A data cleaning method, device and equipment and storage medium

Abstract

The application discloses a data cleaning method and device, equipment and a storage medium. The method comprises the following steps: determining the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleaning strategy; performing hierarchical cleaning on the logical blocks based on the target cleaning strategy of each logical block until the storage medium to be cleaned is completely cleaned; wherein the target cleaning strategy is one of a first strategy for full cleaning, a second strategy for partial cleaning and a third strategy without data cleaning. In this way, by performing hierarchical cleaning on each logical block of the partition of the storage medium to be cleaned based on the first strategy, the second strategy and the third strategy, a targeted cleaning strategy is provided for different functional logical blocks of the storage medium to be cleaned, so that the data cleaning efficiency is improved while the data security is taken into account, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of data cleaning technology, and in particular to a data cleaning method, apparatus, device and storage medium. Background Technology

[0002] Computing networks, which involve the collaboration of technologies such as cloud computing, edge computing, and mass computing, are becoming the mainstream of computing technology today. Bare metal servers are an important component of the infrastructure layer of computing networks, and more and more users are renting bare metal servers through the network to provide computing power services. When users cancel their bare metal server rentals, user data such as user files and usage records will remain to varying degrees.

[0003] In current technologies, disk cleanup methods typically involve either a full cleanup or cleaning only the disk's metadata. A full cleanup involves repeatedly overwriting all areas of the disk to reset the data, achieving data erasure. However, this method is time-consuming and inefficient for cleaning the entire disk. While cleaning only the disk's metadata can improve cleanup speed, the actual file data stored on the disk remains intact. Some residual data can be recovered using methods such as partition table data recovery, posing a certain security risk to user data. Summary of the Invention

[0004] In view of this, embodiments of this application provide a data cleaning method, apparatus, device, and storage medium, aiming to improve the efficiency and security of data cleaning.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a data cleaning method, the method comprising:

[0007] Determine the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy;

[0008] Based on the target cleanup strategy for each logical block, the logical blocks are cleaned up in different levels until the storage medium to be cleaned up is completely cleaned up.

[0009] The target cleanup strategy is one of the following: a first strategy for full cleanup, a second strategy for partial cleanup, and a third strategy for no data cleanup.

[0010] In the above scheme, determining the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy includes:

[0011] The starting address and type of each logical block are determined based on the block descriptor of each partition of the storage medium to be cleaned.

[0012] The target cleanup strategy is determined based on the type of the logical block;

[0013] Accordingly, the logical blocks are cleaned in a hierarchical manner based on the target cleanup strategy for each logical block, including:

[0014] Based on the target cleanup strategy and the starting address of the logical block, the logical block is cleaned up.

[0015] In the above scheme, the hierarchical cleaning of the logical blocks based on the target cleaning strategy of each logical block includes:

[0016] If the type of the logical block is an i-node bitmap or a logical block bitmap, then the i-node bitmap or logical block bitmap will not be cleaned based on the third strategy;

[0017] If the type of the logical block is a logical block, then partial cleanup is performed based on the second strategy and the starting address of the logical block;

[0018] If the type of the logical block is the i-node bitmap, the logical block bitmap, or a logical block other than the logical block, then a full cleanup is performed based on the first strategy and the starting address of the corresponding logical block.

[0019] In the above scheme, the logical block types include directory blocks and data blocks. The second strategy instructs a full cleanup of the directory blocks and a cleanup of the data blocks based on a preset cleanup ratio. The partial cleanup based on the second strategy and the starting address of the logical block includes:

[0020] The type of the logical block is determined based on the block descriptor;

[0021] If the logical block is a directory block, then a full cleanup is performed based on the starting address of the directory block;

[0022] If the logical block is a data block, then the target data block of the same file is cleaned up based on the starting address according to the preset cleanup ratio and file identifier.

[0023] In the above scheme, the step of cleaning up the target data blocks corresponding to the same file based on the starting address according to the preset cleaning ratio and file identifier includes:

[0024] Based on a hash table and file identifiers, the starting address of at least one data block belonging to the same file is determined; wherein, the hash table is used to represent the starting address of at least one data block corresponding to each file identifier;

[0025] Based on the block division rules, the starting addresses of at least one data block of the same file are divided to determine the target data block corresponding to the same file; the target data block corresponding to the same file includes each block after the same file is divided and the starting address corresponding to each block;

[0026] Based on the preset cleanup ratio, the target data blocks corresponding to the same file are cleaned up according to their starting addresses.

[0027] In the above scheme, the step of cleaning the target data blocks corresponding to the same file based on the preset cleaning ratio and the starting address includes:

[0028] Based on the starting address of each block in the target data block and the preset cleanup priority, the cleanup priority of each block is determined;

[0029] Based on the cleanup priority of each block and the preset cleanup ratio, the target data blocks corresponding to the same file are cleaned up according to their starting addresses.

[0030] In the above scheme, the segmentation rule is a variable-length segmentation rule. The step of dividing the starting address of at least one data block of the same file based on the segmentation rule to determine the target data block corresponding to the same file includes:

[0031] Based on the variable-length content segmentation rule, the starting address of at least one data block of the same file is divided to determine the target data block corresponding to the same file; wherein, the size of each block in the target data block is close to a normal distribution.

[0032] The method in the above scheme further includes:

[0033] Based on the directory information in the directory block and each of the file identifiers, the starting address of at least one data block corresponding to each of the same files is determined; the directory information includes the starting address of the data block corresponding to each file identifier.

[0034] Based on the file identifiers and hash algorithms, generate hash values ​​corresponding to the file identifiers;

[0035] A hash table is generated based on the hash value and the starting address of the data block corresponding to each of the same files.

[0036] In the above scheme, before performing partial cleanup based on the second strategy and the starting address of the logical block, the method further includes:

[0037] Obtain the cleanup level parameters;

[0038] The cleaning ratio is determined based on the cleaning level parameters.

[0039] The method in the above scheme further includes:

[0040] In response to the unsubscribe request, data cleanup of the storage medium to be cleaned is initiated;

[0041] After confirming that the storage medium to be cleaned has been cleaned, the storage medium to be cleaned is switched to a first state where it can be ordered.

[0042] During the data cleaning process, the storage medium to be cleaned is in a second state where it cannot be ordered.

[0043] The method in the above scheme further includes:

[0044] Obtain the partition table of the storage medium to be cleaned, wherein the partition table includes a globally unique identifier partition table or a master boot record partition table;

[0045] The partition table is fully cleaned based on the first strategy.

[0046] Secondly, embodiments of this application provide a data cleaning apparatus, comprising:

[0047] The determination module is used to determine the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy.

[0048] The cleaning module is used to perform hierarchical cleaning of the logical blocks based on the target cleaning strategy of each logical block until the storage medium to be cleaned is completely cleaned; wherein, the target cleaning strategy is one of a first strategy for full cleaning, a second strategy for partial cleaning, and a third strategy for not cleaning data.

[0049] Thirdly, embodiments of this application provide a data cleaning device, including: a processor and a memory for storing a computer program capable of running on the processor, wherein...

[0050] The processor is configured to execute the steps of the method described in the first aspect when running a computer program.

[0051] Fourthly, embodiments of this application provide a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0052] The technical solution provided in this application embodiment determines the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy; based on the target cleanup strategy of each logical block, the logical blocks are cleaned in a hierarchical manner until the storage medium to be cleaned is completely cleaned; wherein, the target cleanup strategy is one of a first strategy for full cleanup, a second strategy for partial cleanup, and a third strategy for not cleaning data. Thus, by using the first, second, and third strategies to clean each logical block of the storage medium partition to be cleaned, hierarchical cleanup of each logical block can be achieved, that is, targeted cleanup strategies are provided for logical blocks with different functions of the storage medium to be cleaned, thereby improving data cleanup efficiency while ensuring data security and enhancing the user experience. Attached Figure Description

[0053] Figure 1 A flowchart illustrating the data cleaning method provided in this application embodiment;

[0054] Figure 2 This is a schematic diagram of a data cleaning method based on related technologies in an application example of this application;

[0055] Figure 3 This is a schematic diagram of a data cleaning method in an application example of this application;

[0056] Figure 4 This is a schematic diagram illustrating the cleanup method for each block in an application example of this application;

[0057] Figure 5 This is a schematic diagram illustrating the creation of a LAB table in an application example of this application;

[0058] Figure 6 This is a schematic diagram illustrating file data block segmentation and cleanup in an application example of this application;

[0059] Figure 7 This is a schematic diagram of the data cleaning device according to an embodiment of this application.

[0060] Figure 8 This is a schematic diagram of the data cleaning device according to an embodiment of this application. Detailed Implementation

[0061] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0063] This application provides a data cleaning method that can be applied to data cleaning devices, such as... Figure 1 As shown, the method includes the following steps:

[0064] Step 110: Determine the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy.

[0065] Here, storage media refers to the carriers on which data is stored. This includes, but is not limited to, at least one of the following: disk, floppy disk, optical disk, DVD, hard disk, flash memory, USB flash drive, CF card, SD card, MMC card, SM card, Memory Stick, and xD card. Storage media includes solid-state drives (SSDs) and / or hard disk drives (HDDs). Whether SSD or HDD, storage media is typically partitioned. The most important purpose of partitioning is to facilitate data management. Data or software can be categorized and stored in different partitions based on their type, frequency of use, purpose, or other attributes, which helps maintain overall performance and reliability.

[0066] For example, taking a hard drive as an example, there are two types of hard drive partitioning: 1. MBR (Master Boot Record Partition Table) format, which is a special boot sector located at the beginning of the hard drive; it supports a maximum hard drive size of 2TB and a maximum of 4 partitions. 2. GPT (GUID Partition Table) format. GPT stands for GUID (globally unique identifier) ​​partition table. It is a disk organization method that uses UEFI (unified extensible firmware interface) for booting. Unlike other formats, GPT can support 128 partitions and supports much larger hard drive sizes, far exceeding people's needs.

[0067] A disk contains partitions, and each partition contains its own file system. A disk is a storage medium, a partition is a structure built on the disk that allows one disk to appear as several disks, and a file system is a system built on top of these partitions. This system is implemented using a lot of data stored on the disk. The key purpose of a file system is to provide wear and tear leveling for the storage medium, and it allows data to be stored as files. A commonly used file system consists of six parts: boot block, superblock, inode bitmap, logical block bitmap, inodes, and logical blocks.

[0068] The logical blocks here include at least one of the following: the bootstrap block, superblock, inode bitmap, logical block bitmap, inode, and logical block. Then, based on the different types of logical blocks, the target cleanup strategy corresponding to each logical block is determined.

[0069] Step 120: Perform hierarchical cleaning of the logical blocks based on the target cleaning strategy for each logical block until the storage medium to be cleaned is completely cleaned; wherein, the target cleaning strategy is one of a first strategy for full cleaning, a second strategy for partial cleaning, and a third strategy for not cleaning data.

[0070] Here, after determining the target cleanup strategy for each logical block, the logical blocks are cleaned in a tiered manner until the storage medium to be cleaned is completely cleaned. The target cleanup strategy includes one of the following: a first strategy for full cleanup, a second strategy for partial cleanup, and a third strategy for no data cleanup. That is, different functional blocks are cleaned using targeted cleanup strategies, and logical blocks are cleaned in a tiered manner.

[0071] In this way, by using the first, second and third strategies to clean each logical block of the storage medium partition to be cleaned, it is possible to clean each logical block in a hierarchical manner. That is, targeted cleaning strategies are provided for logical blocks of different functions of the storage medium to be cleaned, thereby improving data cleaning efficiency while ensuring data security and enhancing the user experience.

[0072] In some embodiments, determining the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy includes:

[0073] The starting address and type of each logical block are determined based on the block descriptor of each partition of the storage medium to be cleaned.

[0074] The target cleanup strategy is determined based on the type of the logical block;

[0075] Accordingly, the logical blocks are cleaned in a hierarchical manner based on the target cleanup strategy for each logical block, including:

[0076] Based on the target cleanup strategy and the starting address of the logical block, the logical block is cleaned up.

[0077] Here, each logical block has a corresponding block descriptor that describes it, providing overall information about that logical block. By reading the block descriptor, the starting address and type of each logical block can be determined. For example, the address of the first logical block on the disk where the boot block is located can be determined by reading the block descriptor.

[0078] Here, after determining the type and starting address of each logical block, the target cleanup strategy for the logical block is determined according to the type of the logical block. Finally, based on the target cleanup strategy and the starting address of the logical block, the logical blocks are cleaned up in different levels.

[0079] In some embodiments, before cleaning logical blocks based on the target cleanup strategy and the starting address of the logical blocks, a cleanup method using random number generation and overwriting can be used to clean up the logical blocks. That is, after selecting the corresponding target cleanup strategy for different logical blocks, the cleanup method using random number generation and overwriting can be used for each logical block. Here, the random number overwriting method can be used to clean up all logical blocks, or it can be used to clean up some logical blocks. Random number overwriting allows for faster and more secure data overwriting.

[0080] In some embodiments, the hierarchical cleaning of the logical blocks based on the target cleaning strategy for each logical block includes:

[0081] If the type of the logical block is an i-node bitmap or a logical block bitmap, then the i-node bitmap or logical block bitmap will not be cleaned based on the third strategy;

[0082] If the type of the logical block is a logical block, then partial cleanup is performed based on the second strategy and the starting address of the logical block;

[0083] If the type of the logical block is the i-node bitmap, the logical block bitmap, or a logical block other than the logical block, then a full cleanup is performed based on the first strategy and the starting address of the corresponding logical block.

[0084] Here, after determining the type of each logical block, different target cleanup strategies are configured for each logical block based on its function. Taking a disk as an example, a disk logical block includes a boot block, superblock, inode bitmap, logical block bitmap, inodes, and logical blocks. First, the inode bitmap records the usage of all inodes, and the logical block bitmap records the usage of all logical blocks. These two logical blocks store relatively little information and are not cleaned. Therefore, if the type of the logical block is an inode bitmap or a logical block bitmap, then the inode bitmap or logical block bitmap is not cleaned based on the third strategy. Logical blocks contain a lot of complex information that needs to be cleaned. Therefore, if the type of the logical block is a logical block, then partial cleanup is performed based on the second strategy and the starting address of the logical block. If the type of the logical block is the inode bitmap, the logical block bitmap, or other logical blocks besides the logical blocks, such as the boot block and superblock, then full cleanup can be performed based on the first strategy and the starting address of the corresponding logical block.

[0085] In some embodiments, the logical block type includes directory blocks and data blocks, the second strategy instructs full cleanup of the directory blocks and cleanup of the data blocks based on a preset cleanup ratio, and the partial cleanup based on the second strategy and the starting address of the logical block includes:

[0086] The type of the logical block is determined based on the block descriptor;

[0087] If the logical block is a directory block, then a full cleanup is performed based on the starting address of the directory block;

[0088] If the logical block is a data block, then the target data block of the same file is cleaned up based on the starting address according to the preset cleanup ratio and file identifier.

[0089] Here, logical block types are divided into directory blocks and data blocks. The second strategy, which involves partial cleanup of logical blocks, includes instructing a full cleanup of directory blocks and a cleanup of data blocks based on a preset cleanup ratio. For example, the type of logical blocks contained within a logical block is determined based on the block descriptor. If it is a directory block, a full cleanup is performed based on the starting address of the directory block; if the logical block is a data block, the target data blocks of the same file are cleaned based on their starting addresses according to a preset cleanup ratio and file identifier.

[0090] Here, files are stored in data blocks, and file identifiers are used to identify files. Each file has a unique file identifier for easy retrieval. Based on the file identifier and a preset cleanup ratio, files with the same file identifier are identified as target data blocks for the same file, and then cleanup is performed based on their starting addresses.

[0091] In some embodiments, the step of cleaning up target data blocks corresponding to the same file based on the starting address according to a preset cleaning ratio and file identifier includes:

[0092] Based on a hash table and file identifiers, the starting address of at least one data block belonging to the same file is determined; wherein, the hash table is used to represent the starting address of at least one data block corresponding to each file identifier;

[0093] Based on the block division rules, the starting addresses of at least one data block of the same file are divided to determine the target data block corresponding to the same file; the target data block corresponding to the same file includes each block after the same file is divided and the starting address corresponding to each block;

[0094] Based on the preset cleanup ratio, the target data blocks corresponding to the same file are cleaned up according to their starting addresses.

[0095] Here, the hash table is used to represent the starting address of at least one data block corresponding to each file identifier. A file is stored in at least one data block. Since one file identifier corresponds to at least one data block, and the hash table stores the starting address of at least one data block corresponding to each file identifier, it is possible to determine in the hash table how many data blocks and their starting addresses correspond to the same file identifier (i.e., the same file).

[0096] Here, after finding the starting address of the data block corresponding to the same file, the starting addresses of at least one data block of the same file are divided based on the block division rules to obtain the target data block corresponding to the same file. The target data block corresponding to the same file includes each block after the same file is divided and the starting address corresponding to each block.

[0097] In some embodiments, the step of cleaning the target data blocks corresponding to the same file based on the preset cleaning ratio and the starting address includes:

[0098] Based on the starting address of each block in the target data block and the preset cleanup priority, the cleanup priority of each block is determined;

[0099] Based on the cleanup priority of each block and the preset cleanup ratio, the target data blocks corresponding to the same file are cleaned up according to their starting addresses.

[0100] Here, the target data block of the same file consists of various partitioned blocks and their starting addresses. Cleaning this target data block can be performed by determining the cleanup priority of each block based on a preset cleanup priority, and then cleaning the target data block corresponding to the same file based on its starting address according to the cleanup priority and a preset cleanup ratio. For example, the cleanup level of each block can be arranged from low to high according to the size of its starting address, or the initial block of each block (i.e., the block corresponding to the initial logical address in the target data block) can be cleaned first, while other blocks are cleaned uniformly. For example, if the target data block a corresponding to the same file a is divided into three blocks—block 1, block 2, and block 3—according to logical block addresses, the cleanup priority of this target data block a can be divided into three levels in sequence: 1, 2, and 3, or into two levels in sequence: 1, 2, and 3. That is, block 1 is cleaned first, and then blocks 2 and 3 are cleaned simultaneously, using a uniform cleanup method.

[0101] In some embodiments, the chunking rule is a variable-length chunking rule, and the step of dividing the starting addresses of at least one data block of the same file based on the chunking rule to determine the target data block corresponding to the same file includes:

[0102] Based on the variable-length content segmentation rule, the starting address of at least one data block of the same file is divided to determine the target data block corresponding to the same file; wherein, the size of each block in the target data block is close to a normal distribution.

[0103] Here, the variable-length chunking rule for a file is as follows: First, the total storage space of the file is obtained by accumulating its stored data blocks. Then, the length of the data blocks to be divided is defined to be between a specified minimum and maximum value. The minimum value is the block size *a* bytes corresponding to a logical block address, and the maximum value is 10% of the file's storage space. If the file is too small and the maximum value is less than the minimum value, the maximum and minimum values ​​are the same. According to Rabin's principle, a data fingerprint of a fixed-length sliding window *win* can be calculated. When the data fingerprint in the sliding window matches a preset value, that position is the boundary of the chunk. This process is repeated until the entire file is chunked. After segmentation using this method, a file will be divided into variable-length blocks, and the block sizes will tend to follow a normal distribution.

[0104] In some embodiments, the method further includes:

[0105] Based on the directory information in the directory block and each of the file identifiers, the starting address of at least one data block corresponding to each of the same files is determined; the directory information includes the starting address of the data block corresponding to each file identifier.

[0106] Based on the file identifiers and hash algorithms, generate hash values ​​corresponding to the file identifiers;

[0107] A hash table is generated based on the hash value and the starting address of the data block corresponding to each of the same files.

[0108] Here, the directory block stores the index information and directory information of all files, including the file inode, file type, and filename. Here, the filename is the file identifier. The inode can be queried in the directory block to obtain the address of the data block corresponding to each filename (file identifier). However, the same file may correspond to at least one data block (at least one data block address). Therefore, based on the file identifier (filename), the starting address of at least one data block corresponding to the same file identifier is found in the directory block. For example, if the file identifiers (filenames) are 'a' and 'b', the starting addresses of data block 1 and data block 3 corresponding to file identifier 'a' are found in the directory block, and the starting addresses of data block 2 and data block 4 corresponding to file identifier 'b' are found. Based on the file identifiers (filenames) and a hash algorithm, a hash value corresponding to each file identifier (filename) is generated. Thus, based on this hash value and the starting address of the data block corresponding to the same file, the starting address representing the at least one data block corresponding to each file identifier is generated.

[0109] In some embodiments, before performing partial cleanup based on the second strategy and the starting address of the logical block, the method further includes:

[0110] Obtain the cleanup level parameters;

[0111] The cleaning ratio is determined based on the cleaning level parameters.

[0112] Here, the cleanup ratio can also be obtained based on the cleanup level parameter. Before performing partial cleanup based on the second strategy and the starting address of the logical block, the cleanup level parameter can be obtained, and the cleanup ratio can be determined based on this level parameter. For example, the cleanup level parameter can be set to 5, indicating that the cleanup ratio is 50%.

[0113] In some embodiments, the method further includes:

[0114] In response to the unsubscribe request, data cleanup of the storage medium to be cleaned is initiated;

[0115] After confirming that the storage medium to be cleaned has been cleaned, the storage medium to be cleaned is switched to a first state where it can be ordered.

[0116] During the data cleaning process, the storage medium to be cleaned is in a second state where it cannot be ordered.

[0117] Here, before initiating data cleanup of the storage medium to be cleaned, it can be started based on the user's unsubscription request. That is, the cleanup process is automatically triggered after the storage medium to be cleaned completes the "unsubscription," which can avoid the time occupied by starting the "subscription" process based on the "subscription" request. After it is determined that the storage medium to be cleaned has been cleaned up, the storage medium to be cleaned up is switched to a first state where it can be subscribed; wherein, during the data cleanup process, the storage medium to be cleaned up is in a second state where it cannot be subscribed.

[0118] In some embodiments, the method further includes:

[0119] Obtain the partition table of the storage medium to be cleaned, wherein the partition table includes a globally unique identifier partition table or a master boot record partition table;

[0120] The partition table is fully cleaned based on the first strategy.

[0121] Here, the partition tables of the storage media to be cleaned generally include MBR (Master Boot Record) partition tables or GUID (Globally Unique Identifier) ​​partition tables, depending on their partitioning format. For both, a full cleanup is performed as the primary strategy. GUID partition tables (GPT for short; disks using GUID partition tables are called GPT disks) are a relatively new standard for disk partition table structures. Compared to the currently widely used Master Boot Record (MBR) partitioning scheme, GPT provides a more flexible disk partitioning mechanism. GPT uses a 16-byte Globally Unique Identifier (GUID) value to identify the partition type, making partition type conflicts less likely.

[0122] The embodiments of this application will now be described in further detail with reference to application examples. In this application example, the storage medium to be cleaned is a bare metal disk. Based on this, this application example provides an automatic data cleanup method for bare metal server disks.

[0123] To ensure user data security, the existing solution is to clean up disk metadata before deploying user images on bare metal servers. The specific method is as follows: Figure 2As shown, the bare metal management platform sends a "deployment" request, which is allocated to the bare metal server according to the scheduling policy, and the in-memory operating system is started using the Preboot eXecution Environment (PXE). The agent service of the in-memory operating system starts a disk metadata cleanup program, first erasing the signatures of each disk and partition, including device name, file system type, ID, tag, and other information; then it deletes the entire disk partition table, including the sector distribution and partition size of each partition. After the disk metadata cleanup is completed, the damaged local disk file system of the bare metal server will be unable to perform normal read and write operations; the agent service of the in-memory operating system downloads the user image from the remote image service and writes it to the local disk of the bare metal server, thereby realizing the deployment of the new instance.

[0124] Because existing solutions clean up residual user data during instance deployment, and to ensure timely deployment, current disk cleanup strategies only clean up disk metadata to improve speed. However, the actual file data stored on the disk remains intact, and some residual data can be recovered through methods such as partition table data recovery. This poses a certain security risk to user data.

[0125] In addition, the industry-standard disk cleanup method targets all data on the disk, repeatedly overwriting all areas of the disk to reset the data and achieve the purpose of data erasure. However, this method takes a long time to clean up the entire disk.

[0126] As can be seen from the above, the shortcomings of existing bare metal server disk cleanup methods are:

[0127] (1) The bare metal server ordering process begins when a user initiates an "order" request on the bare metal management platform. However, for the bare metal server to be truly usable by the user, the user image must be fully loaded and the operating system successfully booted from the local disk. The existing disk cleanup process is implemented during instance deployment, which consumes a portion of the ordering time, and failure of the disk cleanup process will result in order failure. Long ordering times and order failures both degrade the user experience.

[0128] (2) Disk metadata cleanup does not destroy the actual data in the file's storage blocks; it only erases the disk and partition signatures, as well as the file system's partition table. Before the disk formatting operation, residual user data can be recovered using existing technologies, such as restoring the file system's partition table information. Therefore, this method carries the risk of exposing user data and lacks protection for user data security. In contrast, traditional full disk data bit reset is inefficient and time-consuming.

[0129] Therefore, this application example provides a method for cleanup disk data on bare metal servers, ensuring that user data cannot be fully recovered using existing technologies and improving disk cleanup efficiency. The disk cleanup process is automatically triggered after the bare metal server completes the "unsubscription" process, avoiding the time consumed by the "subscription" process. The disk data cleanup solution performs level-based cleanup on partitions, file systems, and files, ensuring data security while also improving data cleanup efficiency compared to traditional full-area disk data overwrite.

[0130] This application example provides a cleanup system and flowchart for bare metal disks, such as... Figure 3 As shown, the current usage of bare metal disks involves first creating partitions on a physical disk. Taking MBR partitioning as an example, a maximum of four partitions can be created. GPT partitioning, however, has no limit on the number of partitions. Figure 3 Taking four partitions as an example, each partition can create its own file system. A common file system layout on disk consists of six parts: boot block, superblock, inode bitmap, logical block bitmap, inodes, and logical blocks. The file system here includes the logical blocks corresponding to each partition of the storage medium to be cleaned mentioned above. The function of each logical block is described below.

[0131] (1) Boot block: The boot block of a disk is used to boot the operating system. It generally exists only in the root file system and is empty in other file systems. The boot block is located at the address of the first logical block of the disk.

[0132] (2) Superblock: The superblock of the disk stores the number of file system data blocks, the number of inodes, and records of free status.

[0133] (3) i-node bitmap: records the usage of all i-nodes.

[0134] (4) Logic block bitmap: records the usage of all logic blocks.

[0135] (5) Inode: The inode records the file's metadata, including the file size, permissions, and pointers to index data blocks. This part stores important information such as the logical blocks corresponding to the file data.

[0136] (6) Logical Blocks: Logical blocks are divided into directory blocks and data blocks. The directory block stores file directory information, including file inodes, file types, and filenames. The directory block stores index information for all files and records file data blocks.

[0137] (7) Partition Table: The MBR partition table follows the boot block and is located at the second logical block address on the disk; the GPT partition table exists in multiple locations on the disk. This area is defined by the GPT header and generally occupies sectors LBA (Logical Block Address) 2 to LBA33, where each partition occupies 128 bytes in the sector, called a partition table entry. The GPT header contains a pointer to the partition table, with logical block address LBA1.

[0138] Based on the above structure, in the application example of this application, each logical block has its corresponding target cleanup strategy. The target cleanup strategy includes one of the following: a first strategy for full cleanup, a second strategy for partial cleanup, and a third strategy for no data cleanup. The detailed allocation is as follows:

[0139] (1) For i-node bitmaps and logic block bitmaps, the i-node bitmaps or logic block bitmaps are not cleaned up based on the third strategy.

[0140] Since the information stored in the innode bitmap and logical block bitmap is relatively small, they do not need to be cleaned.

[0141] (2) For logical blocks, partial cleanup is performed based on the second strategy and the starting address of the logical block. The detailed cleanup process is described below.

[0142] (3) For the i-node bitmap, logical block bitmap, and logical blocks other than logical blocks, namely the bootstrap block, superblock, and i-node, perform a full cleanup based on the first strategy, i.e., a 100% full cleanup. Figure 3 The i-node block, the bootstrap block, and the superblock are shown.

[0143] like Figure 3 As shown, the cleanup steps for this bare metal disk are as follows:

[0144] 1. The bare metal management platform automatically triggers the "cleanup" process after initiating an "unsubscribe" request.

[0145] After the bare metal server completes the resource rollback and the "unsubscribe" process, it will automatically trigger the "cleanup" process. At this time, the bare metal server will be in a "cleanup" state, which is an unsubscribeable state, so it will not affect the user's subscription process. That is, in response to the unsubscribe request, the data cleanup of the storage medium to be cleaned is initiated; during the data cleanup process, the storage medium to be cleaned is in a second state that cannot be subscribed to.

[0146] 2. Determine the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy; perform hierarchical cleanup on the logical blocks based on the target cleanup strategy for each logical block until the storage medium to be cleaned is completely cleaned. Start the disk data cleanup program and pass in the cleanup level parameters. After the "cleanup" process is triggered, the bare metal server starts the memory operating system using the startup execution environment. The agent service in the memory operating system starts the disk data cleanup program and passes in the cleanup level parameters; for example, the cleanup level parameter here is 5, which means overwriting 50% of the data blocks of each file. In addition, when cleaning the bare metal disk, a cleanup method that generates random numbers and overwrites each logical block of the disk can be used to clean up the disk.

[0147] The following is based on Figure 3 Taking partition 1 of the bare metal disk as an example, the specific cleanup steps for the logical blocks corresponding to partition 1 are described, such as... Figure 4 As shown, it includes the following steps:

[0148] Step 401: Identify and record all disks and partition information of the bare metal server.

[0149] The bare metal management platform determines the disk's partition format based on the bare metal disk's partition format, whether it is GPT or MBR partition format.

[0150] Step 402: Determine the starting address and type of each logical block based on the block descriptor.

[0151] The starting address and type of each logical block corresponding to each partition are identified based on the block descriptor, including the starting logical block address of logical blocks such as boot block, superblock, inode, and logical blocks (including directory blocks and data blocks). The types of logical blocks include boot block, superblock, inode, and logical blocks. The target strategy includes one of the following: a first strategy for full cleanup, a second strategy for partial cleanup, and a third strategy of not performing data cleanup.

[0152] Step 403: Create the LAB hash table.

[0153] Logical blocks are divided into directory blocks and data blocks. Before cleaning up the data blocks, it is necessary to establish an LBA hash table that represents the starting address of at least one data block corresponding to each file name.

[0154] Identify directory blocks and the index information of all files recorded within them, and build an index to sequentially record all data blocks associated with each file. For example... Figure 3As shown, the directory block stores file directory information, including the file inode, file type, and filename. For example, the filename "a.txt" corresponds to inode 1, which stores the logical address of the data block corresponding to the filename "a.txt". By traversing all files of type "directory" in the directory block information, we find files of type "file", then query their inodes to obtain the logical block address of their data block, and build a hash table to store the corresponding logical block addresses of data blocks of type "file". The hash value is stored based on the filename. Therefore, files of type "directory" are not recorded in the LBA hash table. After searching all directories in the directory block, an LBA hash table can be built (e.g., ...). Figure 5 ). Figure 5 The filenames in the directory block are named a, b, c, e, and f, corresponding to inodes 3, 4, 5, 6, 7, and 8. Based on the inodes corresponding to each filename, the logical address of the data block corresponding to each file is retrieved. This creates... Figure 5 The LBA hash table on the right is used to represent the logical block address of at least one data block corresponding to the same file. Figure 5 The hash table on the left is a hash table compiled in related technologies. Although it also records the files corresponding to each data block, as can be seen from the comparison between the left and right sides, it cannot clearly show how many data blocks file a corresponds to and the logical starting address of the data blocks.

[0155] Step 404: Clean the target data blocks for each file based on content variable length chunking and cleanup level parameters.

[0156] After establishing the LBA hash table, each file needs to be divided into blocks based on variable-length content rules. This means dividing the file by the starting address of at least one data block according to the block division rules to determine the target data blocks corresponding to the same file. The target data blocks for the same file include each block after the file is divided and its corresponding starting address. The block division rules here are variable-length content block division rules.

[0157] like Figure 6As shown, for a file, taking file 'a' as an example, first, the total storage space of the file is obtained by accumulating its stored data blocks. Then, the length of the data blocks to be divided is defined to be between a specified minimum and maximum value. The minimum value is the block size 'a' bytes corresponding to a logical block address, and the maximum value is 10% of the file's storage space. If the file is too small and the maximum value is less than the minimum value, the maximum and minimum values ​​are the same. According to the Rabin principle, a data fingerprint of a fixed-length sliding window 'win' can be calculated. When the data fingerprint in the sliding window matches a preset value, that position is the boundary of the block. This process is repeated until the entire file is divided into blocks. After segmentation using the above method, file 'a' will be divided into blocks of variable length, and the block size will tend to a normal distribution (e.g., ...). Figure 6 File a is divided into 9 blocks based on logical block addresses. Here, file a, divided into 9 blocks, is the target data block.

[0158] After identifying the target data block, it is also necessary to determine the cleanup level of the target data block; that is, based on the starting address of each block in the target data block and the preset cleanup priority, the cleanup priority of each block is determined.

[0159] The cleanup priority can be arranged from low to high based on the size of the starting address of each block, or the initial block of each block (i.e., the block corresponding to the initial logical address in the target data block) can be cleaned first, while other blocks are cleaned uniformly. For example, such as... Figure 6 As shown, the block location, i.e., the cleanup level, tends to an exponential distribution (e.g., Figure 6 The index distribution is based on the starting address of each block in the file, from low to high. That is, the cleanup priority decreases from 1 to 9, or as... Figure 6 The dashed section in file a prioritizes cleaning the block corresponding to the initial logical address in the target data block, while other blocks are cleaned uniformly. Regardless of the method chosen, the blocks at the beginning of file a (the target data block) will be prioritized for cleaning. After determining the cleaning priority, at least one data block corresponding to each file can also be cleaned based on the cleaning level parameter.

[0160] For logical blocks, they are divided into directory blocks and data blocks. If the logical block type is a data block, then according to the cleanup level parameter 5, 50% of the data blocks and file identifiers of each file are cleaned. For the same file, the target data blocks are cleaned based on their starting address. Here, the file identifier is the file name, such as... Figure 3As shown, the file name is "a.txt". The level parameter of 5 means that 50% of the total capacity of data block 1 + data block 2 + data block 3 occupied by the file name "a.txt" will be cleaned up. Data block cleanup can also be based on a preset ratio, which can be set by the user according to their needs.

[0161] After establishing the cleanup level and its parameters, file blocks are selected based on the cleanup level, and the target data blocks are cleaned according to the cleanup level parameters. Specifically, for file data blocks recorded in the LBA hash table, the file is divided into multiple irregular blocks, one file at a time, using a variable-length content-based block partitioning method. 50% of these blocks are selected, and random numbers are generated to overwrite the addresses of these selected logical blocks, thus completing the file data block cleanup. This method selectively cleans blocks based on a strategy, improving cleanup efficiency compared to traditional full-block overwrite cleanup methods while ensuring data integrity. Each file is partitioned into variable-length content blocks, and based on the cleanup level, a portion of the corresponding data blocks are selected. Random numbers are generated to overwrite the disk sectors of the corresponding data blocks to achieve data unrecoverability.

[0162] Step 405: Perform a full cleanup of the directory blocks.

[0163] The type of the logical block is determined based on the block descriptor. If the logical block is a directory block, a full cleanup is performed based on the starting address of the directory block. Since the directory block stores the index information of all files, the starting positions E and F of the logical block address of the directory block on the disk can be determined by reading the block descriptor. Random numbers are generated to completely overwrite logical block addresses E to F, totaling (FE) × a bytes. That is, a full 100% cleanup is performed. Figure 3 The directory block shown.

[0164] Step 406: Clean up the bootstrap block, superblock, and inodes based on the first strategy.

[0165] For the boot block, the address of the first logical block on the disk can be determined by reading the block descriptor. Taking a logical block address of size 'a' bytes as an example, a random number is generated to completely overwrite 'a' bytes of the logical block.

[0166] For the superblock, by reading the block descriptor, it can be known that the boot block is located at the starting position A and B of the logical block address on the disk. A random number is generated to completely overwrite the logical block addresses A to B, totaling (BA) × a bytes.

[0167] For the inode, by reading the block descriptor, it can be determined that the inode block is located at the starting position C and D of the logical block address on the disk. A random number is generated to completely overwrite logical block addresses C to D, totaling (DC) × a bytes. This represents a full 100% cleanup. Figure 3The guide block, superblock, and i-node are shown.

[0168] Step 407: Repeat steps 402-405 to complete the cleanup of all partitions and delete the disk partition table.

[0169] The entire disk's partition table is deleted, including information such as sector distribution and partition size for each partition. For bare metal disks, the partition table is fully cleaned based on the first strategy. Specifically, for MBR partition tables, which immediately follow the boot block, the second logical block address is erased using a random number; for GPT partition tables, which exist in multiple locations on the disk, after identifying the GPT header, the subsequent 33 logical block addresses are completely overwritten using a random number.

[0170] 3. After completing the disk cleanup operation, the agent program restores the bare metal server to a ready-to-order state.

[0171] That is, after determining that the storage medium to be cleaned has been cleaned up, the storage medium to be cleaned up is switched to a first state that can be ordered.

[0172] This application example involves an improvement to the bare metal server disk data cleanup solution. Compared to traditional full disk data overwrite, this technology has a significant advantage in improving data cleanup efficiency, and compared to the original disk metadata cleanup, this technology has a significant advantage in improving data security.

[0173] In summary, this technology offers the advantages of both high efficiency and high security, ensuring the security of user data on bare metal servers, reducing the time required for the "ordering" process, and comprehensively improving the user experience. Furthermore, public cloud bare metal vendors typically offer resource cleanup functions, thus this technology has broad market application prospects.

[0174] To implement the method of this application embodiment, this application embodiment also provides a data cleaning device, which corresponds to the above-described cleaning device method. The steps in the above-described data cleaning device method embodiment are also fully applicable to this data cleaning device embodiment. For example... Figure 7 As shown, the data cleaning device 700 includes a determining module 710 and a cleaning module 720. The determining module 710 determines the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleaning strategy. The cleaning module 720 performs hierarchical cleaning of the logical blocks based on the target cleaning strategy for each logical block until the storage medium to be cleaned is completely cleaned. The target cleaning strategy is one of a first strategy for full cleaning, a second strategy for partial cleaning, and a third strategy that does not perform data cleaning.

[0175] In some embodiments, the determining module 710 is further configured to determine the starting address and type of each logical block based on the block descriptor of each partition of the storage medium to be cleaned;

[0176] The target cleanup strategy is determined based on the type of the logical block;

[0177] The cleaning module 720 is also used to clean the logical block based on the target cleaning strategy and the starting address of the logical block.

[0178] In some embodiments, the cleaning module 720 is further configured to not clean the i-node bitmap or logical block bitmap based on the third strategy if the type of the logical block is an i-node bitmap or a logical block bitmap;

[0179] If the type of the logical block is a logical block, then partial cleanup is performed based on the second strategy and the starting address of the logical block;

[0180] If the type of the logical block is the i-node bitmap, the logical block bitmap, or a logical block other than the logical block, then a full cleanup is performed based on the first strategy and the starting address of the corresponding logical block.

[0181] In some embodiments, the type of the logical block includes a directory block and a data block. The second strategy indicates that the directory block is fully cleaned and the data block is cleaned based on a preset cleanup ratio. The determining module 710 is further configured to determine the type of the logical block based on the block descriptor. The cleanup module 720 is further configured to perform a full cleanup based on the starting address of the directory block if the type of the logical block is a directory block.

[0182] If the logical block is a data block, then the target data block of the same file is cleaned up based on the starting address according to the preset cleanup ratio and file identifier.

[0183] In some embodiments, the determining module 710 is further configured to determine the starting address of at least one data block belonging to the same file based on a hash table and file identifiers; wherein the hash table is used to represent the starting address of at least one data block corresponding to each file identifier; the starting addresses of at least one data block of the same file are divided based on a block division rule to determine the target data block corresponding to the same file; the target data block corresponding to the same file includes each block after the same file is divided and the starting address corresponding to each block; the cleaning module 720 is further configured to clean the target data block corresponding to the same file based on the starting address according to the preset cleaning ratio.

[0184] In some embodiments, the determining module 710 is further configured to determine the cleanup priority of each block based on the starting address of each block in the target data block and a preset cleanup priority; the cleanup module 720 is further configured to clean the target data block corresponding to the same file based on the starting address based on the cleanup priority of each block and the preset cleanup ratio.

[0185] In some embodiments, the segmentation rule is a content variable-length segmentation rule, and the determining module 710 is further configured to divide the starting address of at least one data block of the same file based on the content variable-length segmentation rule to determine the target data block corresponding to the same file; wherein, the size of each block in the target data block is close to a normal distribution.

[0186] In some embodiments, the determining module 710 is further configured to determine the starting address of at least one data block corresponding to each of the same files based on the directory information in the directory block and each of the file identifiers; the directory information includes the starting address of the data block corresponding to each file identifier;

[0187] The data cleaning device also includes a generation module 730, which is used to generate a hash value corresponding to each of the file identifiers based on each of the file identifiers and a hash algorithm;

[0188] A hash table is generated based on the hash value and the starting address of the data block corresponding to each of the same files.

[0189] In some embodiments, the data cleaning apparatus further includes an acquisition module 740 for acquiring cleaning level parameters; and a determination module 710 for determining the cleaning ratio based on the cleaning level parameters.

[0190] In some embodiments, the data cleaning device further includes a startup module 750 and a switching module 760. The startup module 750 is configured to start data cleaning of the storage medium to be cleaned in response to an unsubscribe request. The switching module 760 is configured to switch the storage medium to be cleaned to a first state that can be subscribed to after determining that the storage medium to be cleaned has been cleaned.

[0191] During the data cleaning process, the storage medium to be cleaned is in a second state where it cannot be ordered.

[0192] In some embodiments, the acquisition module 740 is further configured to acquire the partition table of the storage medium to be cleaned, the partition table including a globally unique identifier partition table or a master boot record partition table;

[0193] The cleanup module 720 is also used to perform a full cleanup of the partition table based on the first strategy.

[0194] In practical applications, the determination module 710, cleaning module 720, generation module 730, acquisition module 740, startup module 750, and switching module 760 can be implemented by the processor in the data cleaning device. Of course, the processor needs to run the computer program in the memory to implement its functions.

[0195] It should be noted that the data cleaning device provided in the above embodiments is only illustrated by the division of the above program modules. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the data cleaning device and the data cleaning method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0196] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide a data cleaning device. Figure 8 This is only an exemplary structure of the data cleaning device, not the entire structure; implementation is possible as needed. Figure 8 The structure shown may be part or all of the structure.

[0197] like Figure 8 As shown, the data cleaning device 800 provided in this embodiment includes at least one processor 801, a memory 802, a user interface 803, and at least one network interface 804. The various components of the data cleaning device 800 are coupled together through a bus system 805. It can be understood that the bus system 805 is used to implement communication between these components. In addition to a data bus, the bus system 805 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 8 The general labeled all buses as Bus System 805.

[0198] The user interface 803 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0199] The memory 802 in this embodiment is used to store various types of data to support the operation of the data cleaning device. Examples of such data include any computer program used to operate on the data cleaning device.

[0200] The data cleaning method disclosed in this application can be applied to or implemented by the processor 801. The processor 801 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the data cleaning method can be completed by the integrated logic circuits in the hardware of the processor 801 or by instructions in software form. The processor 801 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 801 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, which is located in the memory 802. The processor 801 reads the information in the memory 802 and, in conjunction with its hardware, completes the steps of the data cleaning method provided in the embodiments of this application.

[0201] In an exemplary embodiment, the data cleaning device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0202] It is understood that memory 802 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0203] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 802 that stores a computer program. The computer program can be executed by the processor 801 of a data cleaning device to complete the steps described in the method of this application embodiment. The computer-readable storage medium can be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0204] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0205] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0206] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data cleaning method, characterized in that, The method includes: Determine the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy; Based on the target cleanup strategy for each logical block, the logical blocks are cleaned up in different levels until the storage medium to be cleaned up is completely cleaned up. The target cleaning strategy is one of the following: a first strategy for full cleaning, a second strategy for partial cleaning, and a third strategy for not cleaning data. The step of determining the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy includes: determining the starting address and type of each logical block based on the block descriptor of each partition of the storage medium to be cleaned; and determining the target cleanup strategy based on the type of the logical block. Accordingly, the logical blocks are cleaned in different levels based on the target cleanup strategy for each logical block, including: cleaning the logical blocks based on the target cleanup strategy and the starting address of the logical blocks; The step of performing hierarchical cleaning of the logical blocks based on the target cleaning strategy for each logical block further includes: if the type of the logical block is an i-node bitmap or a logical block bitmap, then the i-node bitmap or logical block bitmap is not cleaned based on the third strategy; if the type of the logical block is a logical block, then partial cleaning is performed based on the second strategy and the starting address of the logical block; if the type of the logical block is the i-node bitmap, the logical block bitmap, or a logical block other than the logical block, then full cleaning is performed based on the first strategy and the starting address of the corresponding logical block.

2. The method according to claim 1, characterized in that, The logical blocks include directory blocks and data blocks. The second strategy instructs a full cleanup of the directory blocks and a cleanup of the data blocks based on a preset cleanup ratio. The partial cleanup based on the second strategy and the starting address of the logical blocks includes: The type of the logical block is determined based on the block descriptor; If the logical block is a directory block, then a full cleanup is performed based on the starting address of the directory block; If the logical block is a data block, then the target data block of the same file is cleaned up based on the starting address according to the preset cleanup ratio and file identifier.

3. The method according to claim 2, characterized in that, The step of cleaning up target data blocks corresponding to the same file based on the starting address according to a preset cleaning ratio and file identifier includes: Based on a hash table and file identifiers, the starting address of at least one data block belonging to the same file is determined; wherein, the hash table is used to represent the starting address of at least one data block corresponding to each file identifier; Based on the block division rules, the starting addresses of at least one data block of the same file are divided to determine the target data block corresponding to the same file; the target data block corresponding to the same file includes each block after the same file is divided and the starting address corresponding to each block; Based on the preset cleanup ratio, the target data blocks corresponding to the same file are cleaned up according to their starting addresses.

4. The method according to claim 3, characterized in that, The cleaning of target data blocks corresponding to the same file based on the preset cleaning ratio, based on the starting address, includes: Based on the starting address of each block in the target data block and the preset cleanup priority, the cleanup priority of each block is determined; Based on the cleanup priority of each block and the preset cleanup ratio, the target data blocks corresponding to the same file are cleaned up according to their starting addresses.

5. The method according to claim 3, wherein the segmentation rule is a variable-length segmentation rule, and the step of dividing the starting address of at least one data block of the same file based on the segmentation rule to determine the target data block corresponding to the same file includes: Based on the variable-length content segmentation rule, the starting address of at least one data block of the same file is divided to determine the target data block corresponding to the same file; wherein, the size of each block in the target data block is close to a normal distribution.

6. The method according to claim 3, characterized in that, The method further includes: Based on the directory information in the directory block and each of the file identifiers, the starting address of at least one data block corresponding to each of the same files is determined; the directory information includes the starting address of the data block corresponding to each file identifier. Based on the file identifiers and hash algorithms, generate hash values ​​corresponding to the file identifiers; A hash table is generated based on the hash value and the starting address of the data block corresponding to each of the same files.

7. The method according to claim 2, characterized in that, Before performing partial cleanup based on the second strategy and the starting address of the logical block, the method further includes: Obtain the cleanup level parameters; The cleaning ratio is determined based on the cleaning level parameters.

8. The method according to claim 1, characterized in that, The method further includes: In response to the unsubscribe request, data cleanup of the storage medium to be cleaned is initiated; After confirming that the storage medium to be cleaned has been cleaned, the storage medium to be cleaned is switched to a first state where it can be ordered. During the data cleaning process, the storage medium to be cleaned is in a second state where it cannot be ordered.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the partition table of the storage medium to be cleaned, wherein the partition table includes a globally unique identifier partition table or a master boot record partition table; The partition table is fully cleaned based on the first strategy.

10. A data cleaning device, characterized in that, include: The determination module is used to determine the logical blocks corresponding to each partition of the storage medium to be cleaned and the corresponding target cleanup strategy. The determining module is further configured to determine the starting address and type of each logical block based on the block descriptor of each partition of the storage medium to be cleaned; and to determine the target cleanup strategy based on the type of the logical block. The cleanup module is used to perform hierarchical cleanup of the logical blocks based on the target cleanup strategy for each logical block until the storage medium to be cleaned is completely cleaned; wherein, the target cleanup strategy is one of a first strategy for full cleanup, a second strategy for partial cleanup, and a third strategy for not performing data cleanup; the cleanup module is also used to clean the logical blocks based on the target cleanup strategy and the starting address of the logical blocks; the cleanup module is also used to: if the type of the logical block is an i-node bitmap or a logical block bitmap, then not clean the i-node bitmap or logical block bitmap based on the third strategy; if the type of the logical block is a logical block, then perform partial cleanup based on the second strategy and the starting address of the logical block; if the type of the logical block is the i-node bitmap, the logical block bitmap, or a logical block other than the logical block, then perform full cleanup based on the first strategy and the starting address of the corresponding logical block.

11. A data cleaning device, characterized in that, include: A processor and memory for storing computer programs that can run on the processor, wherein, The processor, when running a computer program, performs the steps of the method according to any one of claims 1 to 9.

12. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images