A data copy method, device, system and medium

Abstract

This application discloses a data copying method, apparatus, system, and medium, comprising: reading data from a first storage device to obtain first target data; determining whether second target data exists in the first target data, wherein the second target data is data of a first preset size and a preset format, and the preset format is the data format of the returned data corresponding to the storage space in the first storage device where no data has been written; if the second target data exists in the first target data, filtering the second target data out of the first target data to obtain valid data; and writing the valid data into a second storage device. This allows filtering out useless data and writing only valid data, reducing the amount of data written, lowering copy time, and reducing the frequency of garbage collection triggering, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a data copying method, device, system and medium. Background Art

[0002] PC (personal computer) manufacturers mass-produce PCs and install the operating system on them before they leave the factory. The traditional steps for installing an operating system are as follows: 1. Connect the hard disk to the PC as the local hard disk of the PC. 2. Insert a mobile storage device (such as a CD / USB flash drive) with an operating system installation program into the machine to be installed with the operating system. 3. Execute the operating system installation program on the mobile storage device to install the operating system on the local hard disk of the PC. 4. Perform steps 1-3 on all PCs. This installation procedure was widely used in early PC or personal computer assembly shops, but PC manufacturers usually produce tens of thousands to millions of machines per month. This method has low production efficiency and takes too long to manually intervene in the system installation, making it unsuitable for large-scale production.

[0003] At present, in order to improve production efficiency, PC manufacturers have optimized the production process through data copying. Data copying refers to completely copying and cloning the data on the data source disk (master disk) to the data destination disk (slave disk). The system disk is made with the help of data copying equipment. The steps are as follows: 1. Select a PC and connect the hard disk to the PC as the local hard disk of the PC. 2. Use the traditional method to install the operating system to the local hard disk on the PC in step 1. 3. Remove a local disk from the PC where the operating system is installed in steps 1 and 2, and connect it to the copying machine as the master disk. 4. Connect the slave disk to the copying machine (usually multiple disks are connected at the same time). 5. Use the copying machine to copy the data from the master disk to all the slave disks (the slave disk completely copies and clones the master disk data). 6. Connect the slave disk that completely copies and clones the master disk data to other machines. 7. Repeat 4-6 to complete mass production. However, for the master disk, the data written by the operating system installer to the master disk may be less than 30GB; but for the slave disk, the copy machine will completely copy and clone the entire data of the master disk to the slave disk, resulting in the amount of data written to the slave disk actually equal to the storage space size of the entire master disk. Therefore, this copying method will bring two problems: 1. The amount of data copied to the slave disk is much larger than the amount of data actually written to the master disk, and the increase in the amount of data will cause the copying time to become longer. 2. For the slave disk, the copied data is all valid data, so the slave disk has more valid data. The slave disk that has just left the factory is full of valid data, which causes the garbage collection mechanism of the solid-state drive to be triggered earlier, the amount of valid data moved by garbage collection increases, and the amount of invalid space recovered is relatively small, which increases the time to identify valid data and move valid data. At the same time, the garbage collection mechanism of the solid-state drive will be triggered more frequently, affecting the user experience. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a data copy method, device, system and medium, which can filter useless data, write only valid data, reduce the amount of data written, reduce the copy time, and reduce the frequency of garbage collection triggering, thereby improving user experience. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a data copying method, comprising:

[0006] Reading data from a first storage device to obtain first target data;

[0007] Determine whether there is second target data in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is a data format of returned data corresponding to a storage space in the first storage device where no data is written;

[0008] If the second target data exists in the first target data, filtering the second target data from the first target data to obtain valid data;

[0009] The valid data is written into the second storage device.

[0010] Optionally, reading data from the first storage device to obtain first target data includes:

[0011] Data of a second preset size is read from the first storage device to obtain the first target data.

[0012] Optionally, writing the valid data into a second storage device includes:

[0013] According to the logical block address of the valid data in the first storage device, the valid data is stored in the storage space of the corresponding logical block address in the second storage device.

[0014] Optionally, writing the valid data into a second storage device includes:

[0015] The valid data is written in batches to a plurality of the second storage devices.

[0016] In a second aspect, the present application discloses a data copying device, comprising:

[0017] A data reading module, used for reading data from a first storage device to obtain first target data;

[0018] a data detection module, configured to determine whether second target data exists in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is a data format of returned data corresponding to a storage space in the first storage device where no data is written;

[0019] a data filtering module, configured to filter out the second target data from the first target data if the second target data exists in the first target data, so as to obtain valid data;

[0020] A data writing module is used to write the valid data into the second storage device.

[0021] Optionally, the data reading module is specifically used to:

[0022] Data of a second preset size is read from the first storage device to obtain the first target data.

[0023] Optionally, the data writing module is specifically used to:

[0024] According to the logical block address of the valid data in the first storage device, the valid data block is stored in the storage space of the corresponding logical block address in the second storage device.

[0025] Optionally, the data writing module is specifically used to:

[0026] The valid data is written in batches to a plurality of the second storage devices.

[0027] In a third aspect, the present application discloses a data copying system, characterized in that it comprises: a first storage device, a second storage device and a copying device, wherein the copying device is used to implement the aforementioned data copying method.

[0028] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned data copying method when executed by a processor.

[0029] It can be seen that the present application reads data from the first storage device to obtain the first target data, and then determines whether the second target data exists in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is the data format of the return data corresponding to the storage space in the first storage device where no data is written. If the second target data exists in the first target data, the second target data is filtered out from the first target data to obtain valid data, and the valid data is written into the second storage device. In this way, by determining the format of the data returned from the first storage device and filtering the return data of the storage space in the first storage device where no data is written, useless data can be filtered, and only valid data can be written, thereby reducing the amount of data written, reducing the copying time, and reducing the frequency of garbage collection triggering, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0031] Figure 1 A flow chart of a data copying method disclosed in this application;

[0032] Figure 2 A specific data inspection and filtering schematic diagram disclosed in this application;

[0033] Figure 3A schematic diagram of a specific data copying method disclosed in this application;

[0034] Figure 4 This is a schematic diagram of the structure of a data copying device disclosed in this application;

[0035] Figure 5 This is a structural diagram of a data copy system disclosed in this application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] At present, in order to improve production efficiency, PC manufacturers use data copying equipment to make system disks. The steps are as follows: 1. Select a PC and connect the hard disk to the PC as the local hard disk of the PC. 2. Use the traditional method to install the operating system to the local hard disk on the PC in step 1. 3. Remove a local disk from the PC where the operating system is installed in steps 1 and 2, and connect it to the copying machine as the master disk. 4. Connect the slave disks to the copying machine (usually multiple disks are connected at the same time). 5. Use the copying machine to copy the data from the master disk to all the slave disks (the slave disks completely copy and clone the data of the master disk). 6. Connect the slave disks that completely copy and clone the data of the master disk to other machines. 7. Repeat 4-6 to complete mass production. However, for the master disk, the data written to the master disk by the operating system installation program may be less than 30GB; but for the slave disk, the copying machine will completely copy and clone the entire data of the master disk to the slave disk, resulting in the amount of data written to the slave disk actually equal to the storage space size of the entire master disk. Therefore, this copying method will bring two problems: 1. The amount of data copied to the slave disk is much larger than the amount of data actually written to the master disk, and the increase in the amount of data will cause the copying time to become longer. 2. For the slave disk, the data copied in the past are all valid data, so the valid data of the slave disk increases, and the slave disk that has just left the factory is full of valid data, which causes the garbage collection mechanism of the solid-state drive to be triggered earlier, and the amount of valid data moved by garbage collection increases, and the invalid space recovered is relatively small, which increases the time to identify valid data and move valid data. At the same time, it will trigger the garbage collection mechanism of the solid-state drive more frequently, affecting the user experience. To this end, the present application provides a data copying solution that can filter out useless data, write only valid data, reduce the amount of data written, reduce the copying time, and at the same time reduce the frequency of garbage collection triggering, thereby improving the user experience.

[0038] See also Figure 1As shown, the embodiment of the present application discloses a data copying method, including:

[0039] Step S11: reading data from a first storage device to obtain first target data, wherein the first storage device is a main solid-state hard disk.

[0040] In some other embodiments, the first storage device may also be a storage device with data processing function such as HDD.

[0041] Step S12: Determine whether there is second target data in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is a data format of returned data corresponding to a storage space in the first storage device where no data is written.

[0042] In a specific implementation, data of a second preset size may be read from the first storage device to obtain the first target data.

[0043] The second preset size may be 32KB, 64KB, 128KB, etc.

[0044] In a specific implementation, data of a second preset size may be serially read from the first storage device to obtain first target data of the second preset size; and for each first target data, it is determined whether the second target data exists in the first target data.

[0045] In another specific implementation, multiple data of the second preset size may be read from the first storage device in parallel to obtain the multiple first target data, and the step of determining whether the second target data exists in the first target data may be performed in parallel.

[0046] Furthermore, in a specific implementation manner, this embodiment can determine whether all-0 data or all-F data of the first preset size exists in the first target data.

[0047] Of course, in some other embodiments, it may also be in other formats, such as data such as 0xa5.

[0048] That is, the format of the returned data of the storage space of the solid state drive where no data is written can be configured.

[0049] For example, see Figure 2As shown, this embodiment discloses a specific data check and filtering schematic diagram, the first preset size is 4K Byte, the first storage device returns fixed all-0 data for the storage space where no data is written, and the storage space check range for whether data is written is performed with a 4K Byte size as the granularity. Of course, in some other embodiments, the first preset size can be other values, such as 8K, 16K, etc. Among them, SSD (i.e. Solid State Disk, solid state hard disk).

[0050] Step S13: If the second target data exists in the first target data, the second target data is filtered out from the first target data to obtain valid data.

[0051] Step S14: writing the valid data into a second storage device, wherein the second storage device is a solid state hard disk.

[0052] In a specific implementation, the valid data may be stored in a storage space of a corresponding logical block address in the second storage device according to the logical block address of the valid data in the first storage device.

[0053] It should be noted that after filtering the second target data from the first target data, the first target data is split into multiple valid data blocks. Accordingly, the valid data blocks can be stored in the storage space of the corresponding logical block address in the second storage device according to the logical block address of the valid data block in the first storage device.

[0054] Furthermore, the valid data may be written in batches to multiple second storage devices.

[0055] It can be seen that the embodiment of the present application reads data from the first storage device to obtain the first target data, and then determines whether the second target data exists in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is the data format of the return data corresponding to the storage space in the first storage device where no data is written. If the second target data exists in the first target data, the second target data is filtered out from the first target data to obtain valid data, and the valid data is written into the second storage device. In this way, by determining the format of the data returned from the first storage device and filtering the return data of the storage space in the first storage device where no data is written, useless data can be filtered, and only valid data can be written, thereby reducing the amount of data written, reducing the copy time, and reducing the frequency of garbage collection triggering, thereby improving the user experience.

[0056] See also Figure 3As shown, the embodiment of the present application discloses a specific schematic diagram of a data copying method, in which the solid-state hard disk returns data of a fixed pattern (format) for a storage space where no data has been written; the copying tool runs on the data copying machine, and the data is read from the first storage device, i.e., the mother disk, through the copying tool, the data is filtered, and then the filtered data is written to the second storage device, i.e., the slave disk. The main steps include: 1. The copying machine accesses the hard disk data through the logical block address, and reads data from the first storage device address space in sequence according to a certain size X (such as 32KB / 64KB / 128KB / 256KB / …). Assume that the read data block is LBA_X; 2. Check whether the read data block contains fixed pattern data (such as all 0 data) according to a certain size Y (such as 8KB / 16KB). 3. If the X data block contains fixed pattern data of size Y, split the data block X and filter out the part of the fixed pattern. Assume that the addresses of the split data blocks are LBA_X_0 and LBA_X_1, and the sizes are both Z. 4. Write the data blocks split and filtered in step 3 to the LBA storage space corresponding to all second storage devices according to the LBA (Logical Block Addresses) address on the first storage device. For example, write the Z data blocks of LBA_X_0 and LBA_X_1 to LBA_X_0 and LBA_X_1 of all second storage devices. 5. Repeat steps 1-4 until the storage space of the entire first storage device is traversed.

[0057] See also Figure 4 As shown, the present application discloses a data copying device, comprising:

[0058] A data reading module 11 is used to read data from a first storage device to obtain first target data, wherein the first storage device is a main solid state hard disk;

[0059] A data detection module 12 is used to determine whether there is second target data in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is a data format of returned data corresponding to a storage space in the first storage device where no data is written;

[0060] A data filtering module 13, configured to filter out the second target data from the first target data if the second target data exists in the first target data, so as to obtain valid data;

[0061] The data writing module 14 is used to write the valid data into a second storage device, wherein the second storage device is a solid state hard disk.

[0062] It can be seen that the embodiment of the present application reads data from the first storage device to obtain the first target data, and then determines whether the second target data exists in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is the data format of the return data corresponding to the storage space in the first storage device where no data is written. If the second target data exists in the first target data, the second target data is filtered out from the first target data to obtain valid data, and the valid data is written into the second storage device. In this way, by determining the format of the data returned from the first storage device and filtering the return data of the storage space in the first storage device where no data is written, useless data can be filtered, and only valid data can be written, thereby reducing the amount of data written, reducing the copy time, and reducing the frequency of garbage collection triggering, thereby improving the user experience.

[0063] Wherein, the data reading module is specifically used for:

[0064] Data of a second preset size is read from the first storage device to obtain the first target data.

[0065] In a specific implementation, the data reading module 11 is specifically used to serially read data of a second preset size from the first storage device in sequence to obtain first target data of the second preset size; correspondingly, the data detection module 12 is specifically used to determine, for each first target data, whether the second target data exists in the first target data.

[0066] In another specific embodiment, the data reading module 11 is specifically used to read multiple data of a second preset size from the first storage device in parallel to obtain the multiple first target data; correspondingly, the data detection module 12 is specifically used to concurrently perform the step of determining whether the second target data exists in the first target data.

[0067] The data writing module 14 is specifically configured to store the valid data into a storage space of a corresponding logical block address in the second storage device according to the logical block address of the valid data in the first storage device.

[0068] In a specific implementation, the data writing module 14 is specifically configured to write the valid data in batches into a plurality of second storage devices.

[0069] The data detection module 12 is specifically used to determine whether the first target data contains all-0 data or all-F data of the first preset size.

[0070] See also Figure 5As shown, an embodiment of the present application discloses a data copying system, including a first storage device 21, a copying device 22 and a second storage device 23; wherein the copying device 22 is used to implement the data copying method disclosed in the above embodiment.

[0071] For the specific process of the above data copying method, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0072] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the data copying method disclosed in the aforementioned embodiment.

[0073] For the specific process of the above data copying method, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0074] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0075] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0076] The above is a detailed introduction to a data copying method, device, system and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data copying method, characterized in that: include: Reading data from a first storage device to obtain first target data; Determine whether there is second target data in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is a data format of returned data corresponding to a storage space in the first storage device where no data is written; If the second target data exists in the first target data, the second target data is filtered out from the first target data to obtain valid data; after the second target data is filtered out from the first target data, the first target data is split to obtain data as a plurality of valid data blocks; Writing the valid data into a second storage device; Wherein, writing the valid data into the second storage device comprises: According to the logical block address of the valid data block in the first storage device, the valid data block is stored in the storage space of the corresponding logical block address in the second storage device.

2. The data copying method according to claim 1, characterized in that: The step of reading data from the first storage device to obtain first target data includes: Data of a second preset size is read from the first storage device to obtain the first target data.

3. The data copying method according to claim 1, characterized in that: Writing the valid data into a second storage device comprises: The valid data is written in batches to a plurality of the second storage devices.

4. A data copying device, characterized in that: include: A data reading module, used for reading data from a first storage device to obtain first target data; a data detection module, configured to determine whether second target data exists in the first target data, wherein the second target data is data of a preset format of a first preset size, and the preset format is a data format of returned data corresponding to a storage space in the first storage device where no data is written; a data filtering module, configured to filter out the second target data from the first target data if the second target data exists in the first target data, so as to obtain valid data; after filtering out the second target data from the first target data, the first target data is split, so as to obtain data into a plurality of valid data blocks; A data writing module, used for writing the valid data into a second storage device; The data writing module is specifically used for: According to the logical block address of the valid data block in the first storage device, the valid data block is stored in the storage space of the corresponding logical block address in the second storage device.

5. The data copying device according to claim 4, characterized in that: The data reading module is specifically used for: Data of a second preset size is read from the first storage device to obtain the first target data.

6. The data copying device according to claim 4, characterized in that: The data writing module is specifically used for: The valid data is written in batches to a plurality of the second storage devices.

7. A data copying system, characterized in that: include: A first storage device, a second storage device and a copy device, wherein the copy device is used to implement the data copy method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the data copying method according to any one of claims 1 to 3.

Images