A hard disk data processing method, device and electronic equipment

Abstract

The application discloses a hard disk data processing method and device and electronic equipment, comprising: based on the data characteristics in the first hard disk to be processed, extracting the target data to be migrated of the first hard disk; obtaining the target file system of the first hard disk, the target file system combines the first hard disk and the second hard disk in the same file system; based on the partition corresponding relationship between the second hard disk and the first hard disk of the target file system, migrating the target data to the second hard disk. Based on the target file system, the data of the first hard disk can be migrated to the second hard disk online, the data migration time is shortened, the processing of other business data is not affected in the migration process, and the application can be applied to the hard disk replacement scene, so that the influence on the distributed storage performance is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and more particularly to a hard disk data processing method and device and electronic equipment. BACKGROUND

[0002] At present, distributed storage systems are widely used. In a large-scale data storage scenario, in order to balance performance and cost, a high-performance small-capacity ssd (solid state disk) is used as a distributed storage system (Ceph) OSD (object storage device) metadata cache disk, a low-cost large-capacity hdd (hard disk drive) is used as a data disk, and a single ssd partition is combined with multiple different hdds to form a Ceph OSD to undertake upper-layer services.

[0003] Since the ssd uses flash memory as the storage medium, the number of times of erasing and writing is limited, and the service life is shorter than that of a traditional mechanical disk. When the service life of the ssd is exhausted, multiple osds on the ssd will be in an abnormal state, which seriously affects the normal operation of services. Therefore, when the service life of the ssd cache disk is expected to be exhausted, the Ceph cache disk needs to be replaced in time to ensure the normal operation of the Ceph cluster. However, for a distributed storage system, the number of hard disks to be replaced is large, the steps are cumbersome, and the replacement time is long, which affects the performance of the Ceph. SUMMARY

[0004] Therefore, the present application provides the following technical solutions:

[0005] A hard disk data processing method comprises the following steps:

[0006] Based on the data characteristics in the first hard disk to be processed, target data to be migrated of the first hard disk is extracted;

[0007] A target file system of the first hard disk is obtained, and the target file system combines the first hard disk and the second hard disk in the same file system;

[0008] Based on the partition correspondence relationship between the second hard disk of the target file system and the first hard disk, the target data is migrated to the second hard disk.

[0009] Optionally, the step of extracting the target data to be migrated of the first hard disk based on the data characteristics in the first hard disk to be processed comprises the following steps:

[0010] Based on the data characteristics in the first hard disk to be processed, effective data characteristics are determined;

[0011] Based on the effective data characteristics, the target data to be migrated of the first hard disk is extracted.

[0012] Optionally, further comprising:

[0013] partitioning the second hard disk to obtain at least one second partition;

[0014] matching the at least one first partition corresponding to the first hard disk with the at least one second partition corresponding to the second hard disk to obtain a partition configuration file;

[0015] storing the partition configuration file in the target file system.

[0016] Optionally, the migrating the target data to the second hard disk based on the partition correspondence relationship between the second hard disk and the first hard disk of the target file system comprises:

[0017] calling the partition configuration file in the target file system;

[0018] determining, based on the partition configuration file, a correspondence relationship between a partition corresponding to the target data of the first hard disk and a partition corresponding to the second hard disk;

[0019] migrating the target data to a target partition corresponding to the second hard disk based on the correspondence relationship.

[0020] Optionally, the method further comprises:

[0021] in response to migrating the target data to the second hard disk, obtaining current state information of the first hard disk;

[0022] if the current state information of the first hard disk meets a hard disk rejection condition, backing up data of the first hard disk;

[0023] in response to backing up the data of the first hard disk, replacing the first hard disk with a target hard disk.

[0024] Optionally, the replacing the first hard disk with a target hard disk comprises:

[0025] in response to unmounting a mounting point corresponding to a data processing process of the first hard disk, replacing the first hard disk with a target hard disk.

[0026] Optionally, the method further comprises:

[0027] migrating target data in the second hard disk to the target hard disk and updating a partition configuration file in a target file system of the target hard disk;

[0028] in response to completing the updating of the partition configuration file, mounting the data processing process to the target hard disk, so that the target hard disk performs data processing based on the data processing process.

[0029] Optionally, the migrating the target data to the second hard disk comprises:

[0030] generating a migration thread corresponding to the target data based on data partition features and data volume of the target data to be migrated;

[0031] migrating the target data to the second hard disk based on the migration thread;

[0032] generating migration progress information based on real-time migrated data volume corresponding to the target data, so that the migration progress information is displayed during the migration of the target data.

[0033] A hard disk data processing apparatus comprises:

[0034] an extraction unit configured to extract target data to be migrated from a first hard disk based on data features in the first hard disk to be processed;

[0035] an acquisition unit configured to acquire a target file system of the first hard disk, the target file system combining the first hard disk and a second hard disk in a same file system;

[0036] a migration unit configured to migrate the target data to the second hard disk based on a partition correspondence relationship between the second hard disk and the first hard disk of the target file system.

[0037] An electronic device comprises:

[0038] a memory configured to store an application program and data generated by running of the application program;

[0039] a processor configured to execute the application program to implement the hard disk data processing method according to any one of the preceding embodiments.

[0040] A readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the hard disk data processing method according to any one of the preceding embodiments.

[0041] According to the technical solution, the application discloses a hard disk data processing method, device and electronic equipment, which comprises the following steps: extracting target data to be migrated from a first hard disk based on data characteristics of the first hard disk; obtaining a target file system of the first hard disk, wherein the target file system combines the first hard disk and a second hard disk in a same file system; and migrating the target data to the second hard disk based on a partition correspondence relationship between the second hard disk and the first hard disk of the target file system. The target file system is used to realize online migration of data from the first hard disk to the second hard disk, thereby shortening the data migration time and not affecting processing of other business data during the migration process. The application can be applied to scenarios such as hard disk replacement, thereby reducing the influence on distributed storage performance. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of the provided drawings.

[0043] Figure 1 A flowchart of a hard disk data processing method provided by the embodiment of the application is shown in the figure.

[0044] Figure 2 A partition matching diagram of a first hard disk and a second hard disk provided by the embodiment of the application is shown in the figure.

[0045] Figure 3 A whole flowchart of cache disk replacement provided by the embodiment of the application is shown in the figure.

[0046] Figure 4 A flowchart of cache disk data segment copying to an hdd backup partition provided by the embodiment of the application is shown in the figure.

[0047] Figure 5 A main data segment local copying diagram provided by the embodiment of the application is shown in the figure.

[0048] Figure 6 A flowchart of hdd backup partition data segment copying to a cache disk partition provided by the embodiment of the application is shown in the figure.

[0049] Figure 7 A structure diagram of a hard disk data processing device provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0050] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0051] In the embodiments of the present application, a hard disk data processing method is provided, which can be applied in distributed data storage, in the application scenario of using ssd (Solid State Disk) as a cache disk, to improve the processing performance of hard disk data, and better apply to data migration, replacement, copying and other processing procedures in the hard disk, thereby improving the performance of distributed storage.

[0052] Referring to Figure 1 A flowchart of a hard disk data processing method provided in the embodiments of the present application is shown. The method can include the following steps:

[0053] S101, based on the data characteristics in the first hard disk to be processed, extracting target data to be migrated in the first hard disk.

[0054] Since the ssd (Solid State Disk) uses flash memory as the storage medium, the number of erasing and writing times is limited, and the service life is shorter than that of the traditional mechanical disk. When the service life of the ssd is exhausted, multiple processes on the ssd will be in an abnormal state, which will seriously affect the normal operation of the business. Therefore, when it is expected that the service life of the ssd cache disk will be exhausted, the ssd cache disk needs to be replaced. Therefore, the first hard disk can be the ssd cache disk in this scenario, and correspondingly, the first hard disk can also refer to other types of hard disks that need to migrate storage data or cache data. The data migration can be data copying.

[0055] The data characteristics in the first hard disk refer to the related attribute characteristics of the data stored or cached in the first hard disk, such as data format characteristics, data distribution characteristics, data attribute field characteristics, and data validity period characteristics. Therefore, the target data to be migrated can be determined according to the data characteristics in the first hard disk, so that the migration of invalid data can be avoided, and the efficiency of data migration can be improved.

[0056] In an embodiment, the extracting of the target data to be migrated in the first hard disk based on the data characteristics in the first hard disk to be processed includes: determining valid data characteristics based on the data characteristics in the first hard disk to be processed; and extracting the target data to be migrated in the first hard disk based on the valid data characteristics.

[0057] The effective data feature refers to a feature of data that can be utilized, such as determining whether current data is valid data through a data time field, and taking the valid data as target data to be migrated. Specifically, the effective data feature can be screened out from the first hard disk according to the data feature in the first hard disk and through a data garbage collection mechanism (gc). The effective data feature can also refer to a feature of data to be migrated, for example, migrating data under a certain topic. The effective data feature can also be a data feature representing continuous data. Further, the data in the first hard disk can be arranged as continuous data through the data fragmentation arrangement, so as to realize migration of only effective data, without full migration of data in the first hard disk, thereby shortening the data migration time.

[0058] S102, obtaining a target file system of the first hard disk.

[0059] S103, based on a partition correspondence relationship between the target file system of the second hard disk and the first hard disk, migrating target data to the second hard disk.

[0060] The target file system combines the first hard disk and the second hard disk in the same file system, that is, the target file system has a multi-disk file system feature. The second hard disk can be one or multiple, that is, the first hard disk can correspond to multiple second hard disks. Specifically, the target file system can be an f2fs file system. The f2fs file system can be designed according to an internal geometry structure and a flash memory management mechanism (FTL). The flash memory storage device has many different attributes, so multiple parameters are added in the design process of the f2fs file system, which are not only used to configure the disk layout, but also can select allocation and cleaning algorithms, and optimize performance (parallel IO improves performance). If the first hard disk is an ssd and the second hard disk is an hdd (Hard Disk Drive), the f2fs file system can combine the ssd and the hdd in the same file system for use, realizing a data tiered storage framework. The f2fs fully considers the problem of ssd life wear leveling, and manages data through a segment (data file space). Each segment is 2MB continuous, and all updates are performed in a remote append manner, which is beneficial to converting random write into sequential write.

[0061] Correspondingly, the partition relationship between the first hard disk and the second hard disk can be stored in the target file system, so as to realize migration of data corresponding to the partition of the first hardware based on the target file system.

[0062] In an implementation, the hard disk data processing method further comprises: partitioning the second hard disk to obtain at least one second partition; matching the at least one first partition corresponding to the first hard disk with the at least one second partition corresponding to the second hard disk to obtain a partition configuration file; and storing the partition configuration file in the target file system.

[0063] In the present application, the hard disk partition refers to dividing the overall storage space of the hard disk into multiple independent areas for storing corresponding data or installing corresponding application programs. Figure 2 Fig. 1 is a schematic diagram of partition matching of a first hard disk and a second hard disk provided by an embodiment of the present application. Figure 2 In Fig. 1, the first hard disk is an ssd, and the second hard disk is an hdd. First, the hdd is partitioned to reserve an hdd data backup area part1 with the same size as the ssd partition, which is used for data copying of the cache disk, i.e., for receiving target data migrated from the corresponding ssd partition. Then, the number of ssd partitions is taken modulo the number of hdd partitions to match the ssd partitions with the multiple reserved hdd partitions, and the corresponding relationship is stored in the corresponding partition configuration file, such as the f2fs.conf file. After generating the partition configuration file, data migration can be performed based on the partition configuration file. Figure 2 In Fig. 1, the remaining hdd partition part2 is used as a data disk of the hdd for storing data. It should be noted that the first partition of the first hard disk can correspond to at least one second partition of the second hard disk, such as in Fig. 1. Figure 2 In Fig. 1, the first partition part1 of the first hard disk ssd0 can correspond to the second partitions hdd0 part1 and hdd4 part1 of the second hard disk. Figure 2 In Fig. 1, osd.0 represents a data processing process mounted to ssd0 part1, and osd.7 represents a data processing process mounted to ssd1 part4, where the data processing process is used for processing data in the hard disk.

[0064] After obtaining the partition configuration file in the target file system, the partition corresponding relationship between the second hard disk and the first hard disk can be obtained based on the partition configuration file, and then the target data of the first hard disk is migrated to the second hard disk. In an implementation, the migration of the target data to the second hard disk based on the partition corresponding relationship between the second hard disk and the first hard disk of the target file system comprises: calling the partition configuration file in the target file system; determining the corresponding relationship between the partition corresponding to the target data of the first hard disk and the partition corresponding to the second hard disk based on the partition configuration file; and migrating the target data to the target partition corresponding to the second hard disk based on the corresponding relationship.

[0065] Specifically, the partition information of the target data of the first hard disk located in the first partition of the first hard disk is obtained, and then the target data of the first hard disk is migrated to the second hard disk according to the partition correspondence relationship, for example, in Figure 2 If the target data in the ssd0 part1 partition needs to be migrated, the corresponding partition of the ssd0 part1 partition in the hdd hard disk is hdd0 part1 or hdd4 part1 according to the partition configuration file, and then the target data in the ssd0 part1 partition can be migrated to hdd0 part1 or hdd4 part1.

[0066] The hard disk data processing method disclosed in the embodiments of the application can be applied to the scenario of replacing the cache disk in a distributed system, and the first hard disk to be processed can be the cache disk to be replaced in the scenario. In an implementation manner, in response to migrating the target data to the second hard disk, current state information of the first hard disk is obtained; if the current state information of the first hard disk meets the hard disk rejection condition, the data of the first hard disk is backed up; and in response to backing up the data of the first hard disk, the first hard disk is replaced with a target hard disk.

[0067] The current state information of the first hard disk refers to the current state information that affects the successful rejection, for example, whether the first hard disk currently has business data being executed, whether the first hard disk exists on the host, and whether the data backup process has been started, etc. If the current state of the first hard disk meets the hard disk rejection condition, in order to ensure the subsequent data recovery or the data recovery in the case of failure or abnormality, the data that has not been migrated can be backed up, and the migration of the target data and the backup of important data can be collectively referred to as data backup. After the data backup, the first hard disk is replaced with the target hard disk.

[0068] Further, the first hard disk is replaced with the target hard disk, including: in response to unmounting the mounting point of the data processing process corresponding to the first hard disk, the first hard disk is replaced with the target hard disk.

[0069] Correspondingly, it also includes: migrating the target data in the second hard disk to the target hard disk, updating the partition configuration file in the target file system of the target hard disk; in response to updating the partition configuration file, mounting the data processing process to the target hard disk, so that the target hard disk performs data processing based on the data processing process.

[0070] When the first hard disk is replaced by the target hard disk, the target data migrated from the first hard disk to the second hard disk needs to be migrated back to the target hard disk. At this time, the target hard disk needs to be partitioned first, so that each partition of the target hard disk matches each partition of the second hard disk, thereby obtaining a partition configuration file, that is, updating the original partition configuration file in the target file system, and then based on the updated partition configuration file, the correspondence between the partitions of the target hard disk and the partitions of the second hard disk is determined, and the target data migrated from the first hard disk to the second hard disk is migrated to the target hard disk. And when the first hard disk is migrated, the mounting point of the data process corresponding to the partition of the first hard disk is unloaded, and when the first hard disk is replaced by the target hard disk, the data processing process is re-mounted to the target hard disk, so that the data processing process does not need to be rebuilt, and the data full recovery of the data processing process is not needed, and the business IO impact of the incremental recovery process is very small, thereby enabling the distributed system cluster to quickly recover.

[0071] In an embodiment of the embodiment of the present application, the migrated data amount can also be statistically analyzed in real time, and the data migration progress can be dynamically displayed. In addition, multi-thread segmentation can be used for data migration or copying, fully utilizing the bandwidth of all mechanical disks in the node, and improving the data migration progress without affecting the normal use of other hard disks. Specifically, the target data is migrated to the second hard disk, including: based on the data partition characteristics and data amount of the target data to be migrated, generating a migration thread corresponding to the target data; based on the migration thread, migrating the target data to the second hard disk; based on the real-time migrated data amount of the target data, generating migration progress information, so that the migration progress information is displayed during the target data migration process.

[0072] Wherein, the data partition characteristics refer to the characteristics of the current target data partition, such as the size of the data amount of the partition, whether the current partition data will affect the data of other partitions, whether there is an association relationship between the data of the partitions, etc. If the target data of each partition is independent, multiple parallel data migration threads can be created to migrate the target data to be migrated in parallel, thereby improving the efficiency of data migration. Further, the number of generated migration threads and the time period for executing data migration can be determined in combination with the current network bandwidth and data transmission parameters, etc. When generating migration progress information, the real-time migrated data amount can be determined by counting the number of data segments of the migrated target data, and the migration progress information is generated, so that the data copying progress can be dynamically displayed.

[0073] The following takes the application scenario of periodically replacing the ssd cache disk in a distributed storage Ceph cluster to ensure normal operation of the business as an example to describe the hard disk data processing method in the embodiments of the present application, wherein the ssd cache disk is the first hard disk in the embodiments of the present application, and the hdd data disk is the second hard disk in the embodiments of the present application. The target file system uses the f2fs file system (Flash-Friendly File System, flash-friendly file system), and the data recycling mechanism (gc) and data migration adjustment of the f2fs file system can be used to perform file system fragmentation and data migration on the storage disk to be replaced for a period of time, reduce the amount of copied data, convert fragmented random data into continuous data, and then reduce the data copying time length, then multi-thread segmented copy the effective data of the f2fs file system in the cache disk to the reserved partition of the plurality of hdd data disks, after the ssd is replaced, the multi-thread segmented copy of the f2fs file system is reproduced to copy back to the new disk, and after remounting, the original data processing process (such as ceph osd) is started, and the cluster is quickly recovered.

[0074] Referring to Figure 3 An overall flowchart of cache disk replacement is provided in the embodiments of the present application. First, the matching initialization of all ssd cache disks and hdd data disks in the node is performed, that is, the partitions of the ssd cache disk are matched with the partitions of the hdd data disk. The hdd can be partitioned during the data processing process of the hard disk (such as ceph osd), and a data backup area with the same size as the ssd partition is reserved, so as to establish the path correspondence between the partitions, that is, the ssd part is matched with a plurality of hdd part1 and a single hdd part2 to form a f2fs file system, which is recorded in the partition configuration file (such as f2fs.conf) for subsequent data migration and backup, for example, hdd part1 is used as a backup partition, and hdd part2 is used as a data partition.

[0075] Then, the OSD (OSD ObjectStore Device, object storage device) undertakes the upper-layer business data, uses the f2fs file system characteristics, modifies the parameters, improves the invalid data recycling (gc) and migration (migration) frequency, and synchronously performs data garbage collection and data migration for a period of time online, arranges the fragmented data on the ssd cache disk, and flushes part of the data on the cache disk to the hdd data disk. Specifically, the data segments in the cache disk are arranged through gc, and the data in the cache disk is flushed to the data disk through migration.

[0076] When the cache disk is removed, first check if the cache disk backup strategy is enabled. If not, do not remove it. Check if the cache disk exists on the host. If not, do not remove it. Check if the cache disk exists in the f2fs configuration file. If not, do not remove it. Set the cluster attribute to noout (set this flag, mon will not delete any OSD from the crush map. When maintaining the OSD, set this flag to prevent CRUSH from automatically rebalancing data when the OSD stops.), norecover (set this flag to disable any cluster recovery operation. When performing maintenance and downtime, set this flag.), prevent data reconstruction; stop the process of the osd corresponding to the cache disk on the host; backup / opt / ceph / f2fs / conf as conf-backup, which is used for rollback if the data copy is abnormal; unload the f2fs file system mount point of the osd corresponding to the cache disk; concurrently execute multiple cache disk partition data segment copy to hdd backup partition.

[0077] Referring to Figure 4 A flowchart of a cache disk data segment copy to an hdd backup partition is provided for the embodiments of the present application. According to the partition configuration file, the cache disk partition and the backup partition uuid path are transmitted, and the f2fs.conf is read to parse the partition matching rule device_map (representing the partition corresponding relationship). Check if the corresponding device is mounted. If mounted, data copy cannot be performed. Open the cache disk partition and the corresponding hdd partition device to obtain the file descriptor for reading and writing.

[0078] Each f2fs file system data allocation layout is obtained through the f2fs superblock, the f2fs segments that need to be copied are counted, the segments are copied to different hdd backup partitions in a multi-threaded manner according to the matching rules during initialization, the disk bandwidth of multiple hdds is fully utilized, the f2fs metadata segments are directly divided and copied to multiple hdd backup partitions using multi-threading; the f2fs data main_segments (main data segments) are divided into valid_segments (valid data segments) and invalid_segments (invalid data segments), wherein the invalid_segments do not contain valid data and do not need to be copied, only the valid_segments need to be copied, the valid_segments position is recorded in the f2fs superblock, the valid segment number is divided, the short board is avoided, and the copying progress is affected, the copied segment number is calculated, and the data copying progress is dynamically displayed. The cache disk partition, hdd data partition, hdd backup partition information, metasegment (metadata segment), data segment (data segment) and respective begin segno (start marker), endsegno (end marker) and segment (data segment) total number are recorded in the backup.conf (f2fs backup file), and the copying backup is completed.

[0079] Referring to Figure 5 A schematic diagram of partial copying of a main data segment is provided for an embodiment of the present application. The main data segment (main_segments) in the ssd includes data segments marked as 0-15, wherein the valid data segments (valid_segments) that need to be copied are data segments 1, 2, 3, 5, 11, 12, 13 and 14. Data segments 1, 2, 3 and 5 can be migrated to the hdd0-part1 partition, and data segments 11, 12, 13 and 14 can be migrated to the hdd4-part1 partition.

[0080] After the data migration and copying are completed, the ssd cache disk can be replaced. That is, the old ssd that is excluded from the cluster is pulled out, and a new ssd is inserted; it is checked whether the cache disk backup strategy is enabled, and if not, the cache disk cannot be replaced; it is checked whether there is an idle ssd on the host, and if not, the cache disk cannot be replaced; the new ssd cache disk is partitioned; and the hdd backup partition data is segmented and concurrently copied back to the cache disk partition.

[0081] Referring to Figure 6A process schematic diagram for hdd backup partition data segment copy back to cache disk partition provided by an embodiment of the application. The process includes: importing new cache disk partition and backup partition uuid path, reading backup.conf to parse backup information; checking whether the corresponding device is mounted, if mounted, data copy back cannot be performed; opening the cache disk partition and the corresponding hdd partition device, obtaining the file descriptor for reading and writing; obtaining the data allocation layout of each f2fs file system through f2fs superblock, copying back the f2fs metadata from the hdd backup partition to the cache disk partition directly through multi-thread; in the f2fs data main_segments, only the valid_segments need to be copied back from the multiple hdd backup partitions to the corresponding segment position of the cache disk partition, without full copy; modifying the corresponding cache disk partition path in the f2fs.conf configuration file, so as to complete the data copy back process.

[0082] After completing the data copy back, the f2fs file system is remounted, the stopped data processing process osd is pulled up, the ceph cluster noout and norecover settings are cancelled, the data fast incremental recovery is waited, the ceph cluster state is recovered to be healthy, and the ceph cache disk replacement is completed.

[0083] Through the above-mentioned manner, the cache disk replacement is completed, the f2fs log structure multi-disk file system characteristics are utilized, the cache disk f2fs file system defragmentation and data migration can be performed online, the data copy time is shortened, and the normal operation of the osd and the business is not affected during the period; the multi-thread segment concurrent copy is adopted, the bandwidth of all mechanical disks in the node is fully utilized, and the use of other ssds is not affected; only the f2fs file system valid data segment needs to be copied, without full copy, the data copy amount is reduced; through the f2fs segment quantity counted during the copy, the data copy progress can be dynamically displayed; the data processing process cephosd does not need to be reconstructed, the ceph osd data full recovery is not needed, the incremental recovery process has very small influence on the business IO, and the ceph cluster can be quickly recovered to be healthy.

[0084] In another embodiment of the application, a structure schematic diagram of a hard disk data processing device is also provided, referring to Figure 7 The device can include:

[0085] The extraction unit 701 is configured to extract target data to be migrated of the first hard disk based on a data feature in the first hard disk to be processed.

[0086] The acquisition unit 702 is configured to acquire a target file system of the first hard disk, and the target file system combines the first hard disk and the second hard disk in the same file system.

[0087] migrate the target data to the second hard disk based on a partition correspondence relationship between the second hard disk of the target file system and the first hard disk.

[0088] The hard disk data processing device disclosed by the embodiment of the present application comprises: an extraction unit configured to extract target data to be migrated of a first hard disk based on data features in the first hard disk; an acquisition unit configured to acquire a target file system of the first hard disk, the target file system combining the first hard disk and a second hard disk in a same file system; and a migration unit configured to migrate the target data to the second hard disk based on a partition correspondence relationship between the second hard disk of the target file system and the first hard disk. The target file system is used to realize online migration of data of the first hard disk to the second hard disk, thereby shortening the data migration time, not affecting processing of other business data during the migration process, and being applicable to scenarios such as hard disk replacement, thereby reducing the impact on the performance of distributed storage.

[0089] In an embodiment, the extraction unit 701 comprises:

[0090] a first determination subunit configured to determine effective data features based on data features in a first hard disk to be processed;

[0091] an extraction subunit configured to extract target data to be migrated of the first hard disk based on the effective data features.

[0092] Optionally, the device further comprises:

[0093] a partition unit configured to partition the second hard disk to obtain at least one second partition;

[0094] a matching unit configured to match at least one first partition corresponding to the first hard disk with at least one second partition corresponding to the second hard disk to obtain a partition configuration file;

[0095] a storage unit configured to store the partition configuration file in the target file system.

[0096] In an embodiment, the migration unit 703 comprises:

[0097] a calling subunit configured to call a partition configuration file in the target file system;

[0098] a second determination subunit configured to determine a correspondence relationship between a partition corresponding to the target data of the first hard disk and a partition corresponding to the second hard disk based on the partition configuration file;

[0099] a migration subunit, configured to migrate the target data to a target partition corresponding to the second hard disk based on the correspondence.

[0100] Further, the apparatus further comprises:

[0101] a state acquisition unit, configured to acquire current state information of the first hard disk in response to migrating the target data to the second hard disk;

[0102] a backup unit, configured to backup data of the first hard disk if the current state information of the first hard disk meets a hard disk rejection condition;

[0103] a hard disk replacement unit, configured to replace the first hard disk with a target hard disk in response to backing up the data of the first hard disk.

[0104] Specifically, the hard disk replacement unit comprises:

[0105] an unloading subunit, configured to replace the first hard disk with a target hard disk in response to unloading a mounting point of a data processing process corresponding to the first hard disk.

[0106] Optionally, the apparatus further comprises:

[0107] a file updating unit, configured to migrate target data in the second hard disk to the target hard disk, and update a partition configuration file in a target file system of the target hard disk;

[0108] a remounting unit, configured to mount the data processing process to the target hard disk in response to completing updating the partition configuration file, so that the target hard disk performs data processing based on the data processing process.

[0109] In an implementation manner, the migration unit is specifically configured to:

[0110] generate a migration thread corresponding to the target data based on a data partition feature and a data amount of the target data to be migrated;

[0111] migrate the target data to the second hard disk based on the migration thread;

[0112] generate migration progress information based on a real-time migrated data amount corresponding to the target data, so that the migration progress information is displayed in the target data migration process.

[0113] It should be noted that the specific implementation of each unit and subunit in the present embodiment can refer to the corresponding content in the foregoing, which will not be described in detail here.

[0114] In another embodiment of the present application, a readable storage medium is also provided, and the readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement each step of the hard disk data processing method according to any one of the above.

[0115] In another embodiment of the present application, an electronic device is also provided, and the electronic device can include:

[0116] a memory, configured to store an application program and data generated by running of the application program;

[0117] a processor, configured to execute the application program to implement the hard disk data processing method according to any one of the above.

[0118] It should be noted that the specific implementation of the processor in the present embodiment can refer to the corresponding content in the foregoing, which will not be described in detail here.

[0119] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0120] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present text can be realized in electronic hardware, computer software or combination of the two. In order to clearly show the interchangeability of hardware and software, the composition and steps of each example have been described in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0121] The steps of the method or algorithm described in combination with the embodiments disclosed in the present text can be directly implemented by hardware, software module executed by a processor, or combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0122] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1.A hard disk data processing method, comprising: extracting target data to be migrated from a first hard disk based on data characteristics in the first hard disk to be processed; obtaining a target file system of the first hard disk, the target file system combining the first hard disk and a second hard disk in a same file system; migrating the target data to the second hard disk based on a partition correspondence relationship of the second hard disk and the first hard disk of the target file system; obtaining current state information of the first hard disk in response to migrating the target data to the second hard disk; performing backup of data of the first hard disk if the current state information of the first hard disk meets a hard disk rejection condition; replacing the first hard disk with a target hard disk in response to performing backup of the data of the first hard disk, the replacing the first hard disk with the target hard disk comprising: replacing the first hard disk with the target hard disk in response to unmounting a mount point of a data processing process corresponding to the first hard disk; migrating the target data in the second hard disk to the target hard disk and updating a partition configuration file in the target file system of the target hard disk; mounting the data processing process to the target hard disk in response to updating the partition configuration file being completed, so that the target hard disk performs data processing based on the data processing process. 2.The method of claim 1, wherein the extracting target data to be migrated from a first hard disk based on data characteristics in the first hard disk to be processed comprises: determining valid data characteristics based on data characteristics in the first hard disk to be processed; extracting target data to be migrated from the first hard disk based on the valid data characteristics. 3.The method of claim 1, further comprising: partitioning the second hard disk to obtain at least one second partition; matching at least one first partition corresponding to the first hard disk with at least one second partition corresponding to the second hard disk to obtain a partition configuration file; and storing the partition configuration file in the target file system. 4.The method of claim 3, wherein the migrating the target data to the second hard disk based on a partition correspondence relationship of the second hard disk and the first hard disk of the target file system comprises: calling the partition configuration file in the target file system; determining a correspondence relationship of a partition corresponding to the target data of the first hard disk and a partition corresponding to the second hard disk based on the partition configuration file; and migrating the target data to a target partition corresponding to the second hard disk based on the correspondence relationship. 5.The method of claim 1, wherein the migrating the target data to the second hard disk comprises: generating a migration thread corresponding to the target data based on data partition characteristics and a data amount of the target data to be migrated; migrating the target data to the second hard disk based on the migration thread; and generating migration progress information based on a real-time migrated data amount corresponding to the target data, so that the migration progress information is displayed during the migration of the target data. 6.A hard disk data processing apparatus, comprising: extracting unit, configured to extract target data to be migrated of the first hard disk based on data features in the first hard disk to be processed; an obtaining unit, configured to obtain a target file system of the first hard disk, the target file system combining the first hard disk and the second hard disk in a same file system; a migrating unit, configured to migrate the target data to the second hard disk based on a partition correspondence relationship of the second hard disk and the first hard disk of the target file system; a state obtaining unit, configured to obtain current state information of the first hard disk in response to migrating the target data to the second hard disk; a backup unit, configured to backup data of the first hard disk if the current state information of the first hard disk meets a hard disk rejection condition; a hard disk replacing unit, configured to replace the first hard disk with a target hard disk in response to backing up the data of the first hard disk; the replacing the first hard disk with the target hard disk comprises: replacing the first hard disk with the target hard disk in response to unmounting a mounting point of a data processing process corresponding to the first hard disk; a file updating unit, configured to migrate the target data in the second hard disk to the target hard disk and update a partition configuration file in the target file system of the target hard disk; a remounting unit, configured to mount the data processing process to the target hard disk in response to updating the partition configuration file, so that the target hard disk performs data processing based on the data processing process. 7.An electronic device, comprising: a memory configured to store an application program and data generated by running of the application program; a processor configured to execute the application program to implement the hard disk data processing method according to any one of claims 1-5.

Images