Disk scanning acceleration method and device for Ext file system

Abstract

The invention relates to the technical field of computer file systems, and discloses a disk scanning acceleration method and device for an Ext file system.The method comprises the steps that the equipment type is obtained, and if the equipment type is single-partition equipment, disk scanning acceleration is directly conducted; if the device type is a multi-partition device, converting the multi-partition device into a single-partition device for disk scanning; when disk scanning is carried out, a device path where the file system partition is located is obtained through system calling, and a to-be-scanned device file in the file system partition is opened; reading key metadata of the file system super block; based on the key metadata, scanning the whole file system from the root directory according to the inode number; after scanning is completed, the occupied memory resources are released, and the scanning file of the file system is closed. By utilizing the metadata structure characteristics of the file system and combining the parallel processing technology, the disk scanning efficiency is remarkably improved, and the system resource consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of computer file systems, and in particular to a disk scanning acceleration method and device for an Ext file system. Background Art

[0002] The Linux Ext series of file systems (such as Ext3 and Ext4) are widely used file systems in Linux systems. They have long been trusted by users for their good stability and high performance. Although the usage of modern file systems such as XFS and Btrfs has gradually increased, Ext4, as a classic and mature file system, is still widely used today for its reliability and applicability. However, disk scanning, as a key task in file system management, shows performance bottlenecks in the following scenarios: Low scanning efficiency: For large-capacity disks, the scanning operation takes a long time, especially when the disk is in poor health or severely fragmented. High resource consumption: During the disk scanning process, the CPU and I / O resource usage is high, affecting the normal operation of other system tasks. Lack of parallel optimization: Traditional disk scanning solutions use linear scanning methods, which do not fully utilize the structural characteristics of the Ext file system (such as logs, block groups, and bitmap information) and the multi-core parallel processing capabilities of modern hardware.

[0003] To solve the above problems, a disk scanning acceleration method and device suitable for Ext file system are proposed, which utilizes the metadata structure characteristics of the file system and combines parallel processing technology to significantly improve the disk scanning efficiency and reduce system resource consumption. Summary of the invention

[0004] The purpose of the present invention is to provide a disk scanning acceleration method and device for an Ext file system, aiming to solve the above-mentioned problem.

[0005] The present invention provides a disk scanning acceleration method for an Ext file system, comprising: Get the device type. If the device type is a single-partition device, directly perform disk scanning acceleration; if the device type is a multi-partition device, convert the multi-partition device into a single-partition device for disk scanning; When performing a disk scan, the device path where the file system partition is located is obtained through a system call, and the device file to be scanned in the file system partition is opened; Read key metadata of the file system superblock; Based on the key metadata, starting from the root directory, the entire file system is scanned according to the inode number; After the scan is completed, the occupied memory resources are released and the scanned files in the file system are closed.

[0006] Preferably, if the device type is a multi-partition device, converting the multi-partition device into a single-partition device for disk scanning includes: If the device type is a multi-partition device, obtain the physical disk's support for parallel I / O; Convert multi-partition devices into single-partition devices for disk scanning based on the physical disk's support for parallel I / O; The support status of the physical disk for parallel I / O includes: physical disk capability, driver support and system configuration status.

[0007] Preferably, the key metadata includes: block information, inode information, descriptor information and feature identification.

[0008] Preferably, the block information includes: block size, number of blocks contained in each block group; The inode information includes: the number of inodes in each block group and the inode size; The descriptor information includes: block group descriptor size and position; The characteristic identifier includes: the enabling state of the sparse super block, the enabling state of the elastic block group and the enabling state of the meta block group.

[0009] Preferably, based on the key metadata, starting from the root directory, the entire file system is scanned according to the inode number, including: Determine the storage offset position and group descriptor table of the inode in the file system based on the inode number; Parse inode information and extract basic attributes of files; Parse the block information, according to the storage mode of the block, use the corresponding method to parse the block information and extract the structure information of the file content or folder; Parse the folder according to its structure information, obtain the inode numbers of the subdirectories and files, and scan the directory tree layer by layer.

[0010] Preferably, determining the storage offset position and the group descriptor table of the inode in the file system according to the inode number includes: Block group calculation: The block group number where the file is located = (inode number - 1) / the number of inodes in each block group; Group Descriptor Block Location: The block where the group descriptor is located = block group number / (block size / block group descriptor size); Offset within group descriptor block: Group descriptor index within block = block group number % (block size / block group descriptor size); Inode index within block group: The index of the inode in the block group = (inode number - 1) % the number of inodes in each block group; Index table position calculation: The inode offset in the block of the index table = inode index / (block size / inode size); Inode physical location calculation: The actual address of inode = (index table offset position + inode offset in the block of the index table) × block size.

[0011] Preferably, the basic attributes of a file include: file type, file size, file permissions and timestamp.

[0012] Preferably, when parsing the inode information and extracting the basic attributes of the file, the process further includes: if there is an inode that is not a file or a directory, filtering it.

[0013] Preferably, the method for parsing block information is a direct / indirect block addressing method and an expansion tree method.

[0014] The present invention also discloses a disk scanning acceleration device for an Ext file system, which is used to apply the above-mentioned disk scanning acceleration method for an Ext file system, comprising: The device acquisition unit is used to acquire the device type. If the device type is a single-partition device, the disk scan is directly accelerated; if the device type is a multi-partition device, the multi-partition device is converted into a single-partition device for disk scanning; The opening unit is used to obtain the device path where the file system partition is located through a system call when performing a disk scan, and to open the device file to be scanned in the file system partition; The read unit is used to read the key metadata of the file system super block; A scanning unit, configured to scan the entire file system starting from the root directory and according to the inode number based on the key metadata; The closing unit is used to release the occupied memory resources and close the scanned files in the file system after the scan is completed.

[0015] Compared with the prior art, the beneficial effect of the present invention is that the present invention utilizes the metadata structure characteristics of the file system and combines parallel processing technology to significantly improve the disk scanning efficiency and reduce system resource consumption. Different device types are treated differently, single-partition devices are directly accelerated to scan, and multi-partition devices are converted to single-partition device scanning, which avoids the complex multi-partition scanning process, greatly improves the scanning speed, saves a lot of time cost, and is particularly suitable for scenarios where file system information needs to be quickly obtained. By obtaining the device path through system calls and opening the device file to be scanned, the file system partition can be accurately located, the key metadata of the file system super block can be read, and based on this, the recursive scan is performed from the root directory according to the inode number, which ensures the comprehensiveness and accuracy of the scan and ensures that all files and directories in the file system can be effectively scanned. After the scan is completed, the occupied memory resources are released in time and the scanned files are closed, which effectively avoids memory leakage and resource waste, so that the system resources can be quickly used by other processes after the scanning task is completed, and the overall system performance and resource utilization are improved. The method is designed for the Ext file system, can be well compatible with existing devices and systems based on the Ext file system, can achieve disk scanning acceleration without large-scale transformation of the file system, reduces the application cost and technical threshold, and is easy to promote and apply in different devices and scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0017] Figure 1 It is a flow chart of a disk scanning acceleration method for Ext file system of the present invention; Figure 2 It is a functional block diagram of a disk scanning acceleration device for Ext file system of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] With the development of science and technology, the emergence of solid-state drives has greatly improved the speed and efficiency of data storage and access, and has gradually become one of the mainstream storage devices. However, the traditional serial I / O scanning method cannot fully utilize the hardware performance, which easily leads to a decrease in user satisfaction.

[0020] On the one hand, despite the increasing popularity of emerging storage technologies such as solid-state drives (SSDs), mechanical hard drives still have a place in providing large-capacity storage solutions and cost-effectiveness. Especially in areas such as data centers, cloud storage, and security monitoring, mechanical hard drives are still irreplaceable. In such application scenarios of big data storage, the time consumption of full disk scanning will be further exacerbated.

[0021] The present invention provides an efficient disk scanning acceleration method, which reduces the disk scanning time and reduces the system resource occupancy rate by combining parallel scanning, metadata analysis and other technologies.

[0022] like Figure 1 As shown, the present invention provides a disk scanning acceleration method for an Ext file system, comprising: Get the device type. If the device type is a single-partition device, directly perform disk scanning acceleration; if the device type is a multi-partition device, convert the multi-partition device into a single-partition device for disk scanning; When performing a disk scan, the device path where the file system partition is located is obtained through a system call, and the device file to be scanned in the file system partition is opened; Read key metadata of the file system superblock; Based on the key metadata, starting from the root directory, the entire file system is scanned according to the inode number; After the scan is completed, the occupied memory resources are released and the scanned files in the file system are closed.

[0023] The present invention significantly improves the speed of disk scanning through parallel scanning technology, especially when processing large-capacity storage devices. At the same time, by accurately parsing the key metadata of the file system super block, the accuracy and integrity of the scan are ensured, and unnecessary scanning overhead is avoided. In addition, the strategy of converting multi-partition devices into single-partition devices for scanning simplifies the scanning process and further improves the scanning efficiency. After completing the scanning task, memory resources are released and scanned files are closed in time, which effectively manages system resources and ensures the stability and reliability of the system. In summary, the present invention provides an efficient and accurate acceleration method for disk scanning of the Ext file system, which is of great significance for improving data storage and access performance.

[0024] To further illustrate the present invention, the professional terms in the present invention are now explained: Index node (inode): A structured record used in the Ext file system to describe the metadata of a file or directory. Each file or directory has an inode number in the file system for unique identification.

[0025] Block: The basic unit of storage allocation in the Ext file system. A block is a collection of multiple clusters, with a size between 1KB and 64KB. By default, the block size is 4KB. By default, the file system can manage up to 232 blocks. However, when the 64-bit feature is specified, a maximum of 264 blocks can be managed.

[0026] Block group: Multiple blocks are combined together to form a block group. Each block group stores the super block, group descriptor, block bitmap and index node bitmap of the block group.

[0027] Super block: The super block records a large amount of information about the entire file system, such as the number of data blocks, the number of inodes, supported features, management information, etc.

[0028] Group descriptors: Each block group in the file system corresponds to a block group descriptor, which is the second content group descriptor in the block group and records the location information of the bitmap and inode table.

[0029] Block bitmap / inode bitmap: The block bitmap tracks the usage of data blocks in a block group. The inode bitmap tracks the usage of inodes in a block group. Figure 1 Each bit is 0 or 1 to indicate the usage of a data block in a block or an inode in the inode table.

[0030] Inode table: used to store the inode information of files.

[0031] In some embodiments of the present application, if the device type is a multi-partition device, the multi-partition device is converted into a single-partition device for disk scanning, including: if the device type is a multi-partition device, obtaining the support status of the physical disk for parallel I / O; converting the multi-partition device into a single-partition device for disk scanning according to the support status of the physical disk for parallel I / O; the physical disk's support status for parallel I / O includes: physical disk capabilities, driver support and system configuration status.

[0032] In this embodiment, when the device type is detected as a multi-partition device, in order to improve the efficiency and accuracy of disk scanning, this technical solution proposes a method for converting a multi-partition device into a single-partition device for disk scanning. The specific steps are as follows: First, the system will detect whether the current device is a multi-partition device. If yes, the system will perform the following steps: Obtaining the physical disk's support for parallel I / O: This step involves evaluating the physical disk's hardware capabilities, whether the currently installed driver supports parallel I / O operations, and whether the system configuration allows parallel I / O operations. Based on the physical disk's support for parallel I / O, convert the multi-partition device to a single-partition device for disk scanning: If the physical disk supports parallel I / O operations, the system will adjust the partition settings to merge multiple partitions into a single partition. This can reduce the number of I / O operations during the disk scan, because scanning a single partition is usually more efficient than scanning multiple partitions. If the physical disk does not support parallel I / O operations, the system may take other optimization measures or follow the traditional multi-partition scanning method.

[0033] Physical disk support for parallel I / O includes but is not limited to the following: Physical disk capacity: including the disk interface type (such as SATA, SAS, etc.), transfer rate, cache size and other hardware characteristics, all of which will affect the performance of parallel I / O. For example, if the SATA interface is SATA 3.0 version, it theoretically supports a higher parallel transfer rate; the number and architecture of processing cores inside the device, storage devices with multi-core architecture have stronger capabilities when processing parallel I / O requests; and the maximum I / O queue depth supported by the device. The greater the depth, the more I / O requests the device can process simultaneously, and the stronger the parallel processing capability.

[0034] Driver support: The disk driver installed in the operating system must support parallel I / O operations. Otherwise, even if the hardware supports it, parallel I / O cannot be implemented. The driver's implementation and optimization of parallel I / O functions are crucial. Some high-performance drivers use technologies such as multithreading or asynchronous I / O to improve parallel processing efficiency. At the same time, the I / O modes supported by the driver, such as Direct I / O, Scatter-Gather I / O, etc., have different effects on parallel I / O performance in different application scenarios.

[0035] System configuration status: The operating system's I / O scheduler settings, system resource allocation policies, and other configurations determine whether the system allows parallel I / O operations and how to optimize these operations. Relevant configuration parameters at the system level, such as the settings of the I / O scheduling algorithm in the kernel parameters, and the selection of different scheduling algorithms such as Deadline and CFQ, will directly affect the system's processing order and efficiency for parallel I / O requests; the system resource allocation policy, including the size of memory allocated to the I / O cache, will also have an important impact on the performance of parallel I / O.

[0036] Through the above steps, the technical solution can intelligently select the most appropriate disk scanning strategy according to the specific situation of the device, thereby improving the efficiency and performance of the disk scanning.

[0037] It can be understood that the present invention can effectively improve the disk scanning efficiency of multi-partition devices on the Ext file system. By converting the multi-partition device into a single-partition device for scanning, the scanning process can be simplified and the time overhead of partition switching and data processing can be reduced. At the same time, the conversion is performed according to the support of the physical disk for parallel I / O, which can make full use of the parallel processing capability of the physical disk and further improve the scanning speed.

[0038] In some embodiments of the present application, the key metadata includes: block information, inode information, descriptor information and feature identification.

[0039] In some embodiments of the present application, the block information includes: block size, the number of blocks contained in each block group; the inode information includes: the number of inodes in each block group, the inode size; the descriptor information includes: block group descriptor size and position; the feature identifier includes: the enable status of sparse super blocks, the enable status of elastic block groups, and the enable status of meta block groups.

[0040] In this embodiment, the definition and content of key metadata are elaborated in detail to ensure efficient management of the file system and accurate storage of data. Key metadata includes several main parts: block information, inode information, descriptor information and feature identification.

[0041] Specifically, block information provides detailed configuration information about data blocks, including block size, i.e., the storage capacity of each data block, and the number of blocks contained in each block group, which determines the overall storage capacity of each block group. With this information, the system can effectively organize and access storage space. Inode information focuses on the index node of the file, including the number of inodes in each block group, which determines the number of files that each block group can support, and inode size, i.e., the storage space occupied by each inode. Inode is a key structure used in the file system to store file metadata, including information such as file permissions, owner, size, and pointers to data blocks. Descriptor information provides detailed information about block group descriptors, including the size and location of block group descriptors. Block group descriptors are data structures used in the file system to describe the status and attributes of each block group, which are crucial to the overall layout and management of the file system. Finally, the feature flags section includes several key system feature states, such as the enablement status of sparse superblocks, the enablement status of elastic block groups, and the enablement status of metablock groups. These feature flags allow system administrators to enable or disable specific advanced features as needed to optimize performance or enhance data reliability. For example, sparse superblocks can reduce superblock redundancy, elastic block groups can provide better data recovery capabilities, and metablock groups can optimize the storage and access efficiency of metadata.

[0042] It can be understood that by accurately identifying and extracting these key metadata, the technical solution can more efficiently organize and process the data information required in the disk scanning process. Specifically, the block information provides the basic layout of the disk blocks for the scanning process, so that the scanning can be carried out in an orderly manner according to the size and distribution of the blocks; the inode information associates the mapping relationship between the file and the disk block, which is the key to locating the file data; the descriptor information ensures that the relevant information of each block group can be accurately found and read during the scanning process; and the feature identifier reflects the specific functions and optimization options of the Ext file system, which helps to optimize the scanning strategy to adapt to different file system configurations.

[0043] The present invention significantly improves the pertinence and efficiency of disk scanning, reduces unnecessary data reading and processing by accurately identifying and extracting key metadata, and thus accelerates the scanning process. At the same time, the technical solution also has strong adaptability and flexibility, and can be intelligently adjusted according to different Ext file system configurations and physical disk characteristics to ensure the efficiency and accuracy of the scanning process.

[0044] In some embodiments of the present application, based on the key metadata, starting from the root directory, the entire file system is scanned according to the inode number, including: determining the storage offset position and group descriptor table of the inode in the file system according to the inode number; parsing the inode information to extract the basic attributes of the file; parsing the block information, and according to the storage method of the block, using a corresponding method to parse the block information, extracting the file content or the structure information of the folder; parsing the folder according to the structure information of the folder, obtaining the inode numbers of the subdirectories and files, and scanning the directory tree layer by layer.

[0045] In this embodiment, starting from the root directory of the file system, it is first necessary to obtain the inode number of the root directory, which is the unique identifier of each file and directory in the file system. Next, use this inode number to determine its storage offset position in the file system and the group descriptor table to which it belongs. The group descriptor table contains metadata information for managing data blocks and inodes in the file system. Once you have the inode number and storage location, you can start parsing the inode information. The inode contains the basic attributes of the file, such as file type, permissions, owner, size, timestamp, and pointers to file data blocks. By parsing this information, the metadata of the file can be extracted. For the extraction of file content, it is necessary to parse the block information associated with the inode. Blocks are the basic units for storing file data in the file system. Depending on how blocks are stored in the file system (e.g., direct blocks, indirect blocks, etc.), different methods will be used to parse these block information. Direct blocks directly point to the physical location where the file content is stored, while indirect blocks point to blocks containing more block pointers, which allows the file system to store larger files. If a folder is currently being parsed, we need to obtain its structural information. The structural information of a folder includes the inode numbers of all subdirectories and files in the folder. Through these inode numbers, each subdirectory can be recursively parsed, repeating the above scanning and parsing process. This process will continue until we have traversed all subdirectories and files, thus completing the scan of the entire directory tree. The whole process requires careful processing of various metadata and data structures in the file system to ensure that the file system hierarchy and content can be accurately reconstructed.

[0046] When parsing a folder based on its structural information, in the traditional folder layout, all sub-item information is stored in a linear structure. However, this linear layout has the problem of low overall traversal performance when processing large directories. For this reason, the Ext3 file system introduces an innovative Hash tree layout that improves performance by combining hash indexes and hierarchical structures. Therefore, when parsing a folder, it is necessary to support both the linear layout and the Hash tree layout to ensure compatibility and avoid potential problems.

[0047] It is understandable that by taking the inode number as the core, the deep traversal and precise scanning of the file system can be achieved. In this process, not only the comprehensiveness and accuracy of the scan are ensured, but also the scanning efficiency is greatly improved. By directly locating the file data according to the inode number, the redundant search and invalid access that may exist in the traditional scanning method are avoided, thereby significantly shortening the scanning time. At the same time, this technical solution can also intelligently process the structural information of the folder, scan the directory tree layer by layer, and ensure that no files or subdirectories are missed, providing strong support for disk maintenance and data recovery.

[0048] In some embodiments of the present application, determining the storage offset position and the group descriptor table of the inode in the file system according to the inode number includes: Block group calculation: The block group number where the file is located = (inode number - 1) / the number of inodes in each block group; Group Descriptor Block Location: The block where the group descriptor is located = block group number / (block size / block group descriptor size); Offset within group descriptor block: Group descriptor index within block = block group number % (block size / block group descriptor size); Inode index within block group: The index of the inode in the block group = (inode number - 1) % the number of inodes in each block group; Index table position calculation: The inode offset in the block of the index table = inode index / (block size / inode size); Inode physical location calculation: The actual address of inode = (index table offset position + inode offset in the block of the index table) × block size.

[0049] In this embodiment, in these calculation methods, "%" is a modulus operator, also called a remainder operator, which is used to calculate the remainder after dividing two numbers.

[0050] Taking "group descriptor index in block = block group number%(block size / block group descriptor size)" as an example, this formula is used to determine the specific index position of the group descriptor in the block where it is located. By dividing the block group number by "block size / block group descriptor size", the quotient is the block number where the group descriptor is located, and the remainder (that is, the result of the "%" operation) is the index of the group descriptor in the block, so that the position of the group descriptor in the block can be accurately located.

[0051] Similarly, in "index of inode in its block group = (inode number - 1) % number of inodes in each block group", the "%" operation calculates the index of the inode in its block group by subtracting 1 from the inode number and dividing it by the number of inodes in each block group, making it easy to quickly find a specific inode in a block group.

[0052] It is understandable that through precise calculation steps, the actual storage location of the inode on the disk can be quickly located, thereby greatly improving the efficiency of disk scanning. Compared with the traditional scanning method, this method reduces unnecessary search and traversal processes, allowing the file system to respond and process faster during maintenance and repair. In addition, this technical solution also optimizes memory usage, reduces performance bottlenecks caused by frequent disk access, and provides a strong guarantee for the stability and efficiency of the Ext file system.

[0053] In some embodiments of the present application, the basic attributes of a file include: file type, file size, file permissions, and timestamp.

[0054] In some embodiments of the present application, when parsing the inode information and extracting the basic attributes of the file, it also includes: if there is an inode that is not a file or a directory, filtering is performed.

[0055] It is understandable that by filtering out non-file or non-directory inodes, the amount of data that needs to be processed during the disk scanning process can be further reduced, thereby improving the scanning efficiency. This filtering mechanism can ensure that only inodes directly related to the file system structure and user data are included in the processing scope, avoiding unnecessary data traversal and analysis. In addition, this beneficial effect is also reflected in the effective use of system resources. By reducing unnecessary processing steps, the CPU and memory usage are reduced, making the file system more smooth and efficient when performing scanning tasks.

[0056] In some embodiments of the present application, the method of parsing block information is a direct / indirect block addressing method and an expansion tree method.

[0057] In this embodiment, the method for parsing block information is a direct / indirect block addressing method and an extended tree method. The direct / indirect block addressing method is a technology for locating and accessing data blocks in a file system. In direct addressing, the file system directly maps the beginning of a file to a physical block on a disk. Usually, the first few blocks of a file can be directly accessed through a direct pointer in a file control block (FCB) or an index node (inode). If the file is large, the direct pointer may not be enough to cover the entire file, and indirect addressing is required. The indirect addressing method extends the addressing capability of the file through one or more indirect blocks. In single indirect addressing, the file control block contains a pointer that points to an indirect block, which contains a pointer to the actual data block. If the file grows further, it may be necessary to use double indirect addressing or even triple indirect addressing, in which the indirect block pointed to by the pointer points to other indirect blocks to form a pointer chain, so that more data blocks can be accessed. The extended tree method, such as a B-tree or a B+ tree, is a more complex data structure for managing data blocks on a disk. A B-tree is a balanced tree that allows fast insertion, deletion, and search of data. In the file system, each node of the B-tree can contain multiple key-value pairs, each corresponding to a data block. The non-leaf nodes of the B-tree contain pointers to child nodes and keys that separate key values, while the leaf nodes contain actual data block pointers or the data itself. The B+ tree is a variant of the B-tree that stores all data records on leaf nodes, while the internal nodes are only used for indexing, which makes the B+ tree more efficient in sequential access. Both methods have their own advantages and disadvantages. The direct / indirect block addressing method is simple and intuitive, but it may cause performance degradation when processing large files because multiple indirect blocks need to be accessed. The extended tree method, especially the B-tree and B+ tree, provides better performance and space utilization when processing large amounts of data and frequent search, insertion, and deletion operations, but it is also more complex to implement.

[0058] It can be understood that by adopting the direct / indirect block addressing method and the extended tree method to parse the block information, the data blocks in the Ext file system can be efficiently accessed and processed. Direct block addressing allows the physical blocks where the file data is located to be quickly located, while the indirect block addressing and extended tree methods provide effective management of large files or complex directory structures. This technical solution combining multiple block addressing methods not only improves the speed of disk scanning, but also enhances the flexibility and scalability of the file system.

[0059] like Figure 2 As shown, the present invention also discloses a disk scanning acceleration device for an Ext file system, which is used to apply the above-mentioned disk scanning acceleration method for an Ext file system, comprising: The device acquisition unit is used to acquire the device type. If the device type is a single-partition device, the disk scan is directly accelerated; if the device type is a multi-partition device, the multi-partition device is converted into a single-partition device for disk scanning; The opening unit is used to obtain the device path where the file system partition is located through a system call when performing a disk scan, and to open the device file to be scanned in the file system partition; The read unit is used to read the key metadata of the file system super block; A scanning unit, configured to scan the entire file system starting from the root directory and according to the inode number based on the key metadata; The closing unit is used to release the occupied memory resources and close the scanned files in the file system after the scan is completed.

[0060] The present invention significantly improves the scanning efficiency of the Ext file system by optimizing the disk scanning process. In particular, when processing a multi-partition device, by converting it into a single-partition device for scanning, the problem of reduced scanning speed caused by multiple partitions in the traditional scanning method is effectively avoided. In addition, the device also realizes fast recursive scanning starting from the root directory by accurately reading the key metadata of the file system super block, further improving the accuracy and efficiency of the scanning. At the same time, after the scanning is completed, the occupied memory resources are released in time and the scanned files are closed, ensuring the stability of the system and the efficient use of resources.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

[0062] The system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0063] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

Claims

1. A disk scanning acceleration method for Ext file system, characterized in that: include: Get the device type. If the device type is a single-partition device, perform disk scanning acceleration directly. If the device type is a multi-partition device, convert the multi-partition device to a single-partition device for disk scanning; When performing a disk scan, the device path where the file system partition is located is obtained through a system call, and the device file to be scanned in the file system partition is opened; Read key metadata of the file system superblock; Based on the key metadata, starting from the root directory, the entire file system is scanned according to the inode number; After the scan is completed, the occupied memory resources are released and the scanned files in the file system are closed.

2. The disk scanning acceleration method for Ext file system according to claim 1, characterized in that: If the device type is a multi-partition device, convert the multi-partition device to a single-partition device for disk scanning, including: If the device type is a multi-partition device, obtain the physical disk's support for parallel I / O; Convert multi-partition devices into single-partition devices for disk scanning based on the physical disk's support for parallel I / O; The support status of the physical disk for parallel I / O includes: physical disk capability, driver support and system configuration status.

3. The disk scanning acceleration method for Ext file system according to claim 2, characterized in that: The key metadata includes: block information, inode information, descriptor information and feature identification.

4. The disk scanning acceleration method for Ext file system according to claim 3, characterized in that: The block information includes: block size, number of blocks contained in each block group; The inode information includes: the number of inodes in each block group and the inode size; The descriptor information includes: block group descriptor size and position; The characteristic identifier includes: the enabling state of the sparse super block, the enabling state of the elastic block group and the enabling state of the meta block group.

5. The disk scanning acceleration method for Ext file system according to claim 4, characterized in that: Based on the key metadata, starting from the root directory, the entire file system is scanned according to the inode number, including: Determine the storage offset position and group descriptor table of the inode in the file system based on the inode number; Parse inode information and extract basic attributes of files; Parse the block information, according to the storage mode of the block, use the corresponding method to parse the block information and extract the structure information of the file content or folder; Parse the folder according to its structure information, obtain the inode numbers of the subdirectories and files, and scan the directory tree layer by layer.

6. The disk scanning acceleration method for Ext file system according to claim 5, characterized in that: Determine the storage offset location and group descriptor table of the inode in the file system based on the inode number, including: Block group calculation: The block group number where the file is located = (inode number - 1) / the number of inodes in each block group; Group Descriptor Block Location: The block where the group descriptor is located = block group number / (block size / block group descriptor size); Offset within group descriptor block: Group descriptor index within block = block group number % (block size / block group descriptor size); Inode index within block group: The index of the inode in the block group = (inode number - 1) % the number of inodes in each block group; Index table position calculation: The offset of inode in the block of the index table = inode index / (block size / inode size); Inode physical location calculation: The actual address of inode = (index table offset position + inode offset within the block of the index table) × block size.

7. The disk scanning acceleration method for Ext file system according to claim 6, characterized in that: Basic file properties, including: file type, file size, file permissions, and timestamp.

8. The disk scanning acceleration method for Ext file system according to claim 7, characterized in that: When parsing the inode information and extracting the basic attributes of the file, it also includes: if there is an inode that is not a file or a directory, it is filtered.

9. The disk scanning acceleration method for Ext file system according to claim 8, characterized in that: The methods for parsing block information are direct / indirect block addressing method and expansion tree method.

10. A disk scanning acceleration device for an Ext file system, used for applying the disk scanning acceleration method for an Ext file system as claimed in any one of claims 1 to 9, characterized in that: include: The device acquisition unit is used to obtain the device type. If the device type is a single-partition device, the disk scan acceleration is performed directly; If the device type is a multi-partition device, convert the multi-partition device to a single-partition device for disk scanning; The opening unit is used to obtain the device path where the file system partition is located through a system call when performing a disk scan, and to open the device file to be scanned in the file system partition; The read unit is used to read the key metadata of the file system super block; A scanning unit, configured to scan the entire file system starting from the root directory and according to the inode number based on the key metadata; The closing unit is used to release the occupied memory resources and close the scanned files in the file system after the scan is completed.

Images