Data writing method and device, computer device and storage medium

Abstract

Embodiments of the present application disclose a data writing method and device, computer equipment and a storage medium, and belong to the technical field of computer storage. In the embodiments of the present application, in the case that the first data and the second data are the same data and the second data has been stored in a target data block in a file system, the first data can be stored repeatedly, and the identification of the target data block can be used as the index information of the first data, so that the target data block can be queried through the index information of the first data, that is, the first data can be read. In this way, the waste of the storage medium can be avoided. Therefore, in the scene of copying a file, the entire data of the file does not need to be stored in the file system again, but only the index information of the original file needs to be stored in the index information of the copied file. In this way, only the data of one file needs to be stored on the storage medium, and therefore the utilization rate of the storage medium is improved.

Description

Technical Field

[0001] This application relates to the field of computer storage technology, and in particular to a data writing method, apparatus, computer device, and storage medium. Background Technology

[0002] To facilitate the management of data stored on storage media in computer devices, file systems are currently built upon the storage space provided by these media. A file system can be understood as a system that organizes the data stored on the storage media. All data on a computer device is written to the file system as files, and subsequently, the data can be retrieved from the storage media through the file system.

[0003] In related technologies, a file system includes data blocks for storing files and inodes for each file. The inode of any file stores the physical address of the data block corresponding to that file. The data block corresponding to that file refers to the data block that stores the data within that file. For any file to be written, unused data blocks are allocated for that file to write the data contained therein. An inode is also allocated for that file. This inode stores the physical address of the previously allocated data block, facilitating subsequent lookup of the data within the file.

[0004] The above-mentioned method of writing to files can easily lead to a large amount of duplicate data being stored on the storage medium, resulting in low utilization of the storage medium. Summary of the Invention

[0005] This application provides a data writing method, apparatus, computer device, and storage medium, which can improve the utilization rate of storage media with mounted file systems. The technical solution is as follows:

[0006] Firstly, a method for writing data is provided, applied to a computer device equipped with a file system. In this method, a first file to be written is obtained. This first file includes first data, which is identical to second data in a second file stored in the file system. The second data is stored in a target data block of the file system. The file system also includes a first inode corresponding to the first file, which stores index information for each piece of data included in the first file. The index information for any piece of data indicates the identifier of the data block storing the corresponding data. Then, the identifier of the target data block is written into the first inode as the index information of the first data.

[0007] Since the first data and the second data are identical, and the second data has already been stored in the target data block of the file system, it is unnecessary to repeatedly store the first data in this embodiment. The identifier of the target data block can be used as the index information of the first data. This allows subsequent lookup of the target data block using the index information of the first data, thereby retrieving the data within the target data block – that is, reading the first data. This avoids wasting storage media. Furthermore, the identifier of the target data block indicates its specific location on the storage media; that is, the identifier can serve as the index information for the data stored within the target data block. Therefore, in file copying scenarios, it is unnecessary to re-store the entire file data into the file system; only the index information of the original file needs to be stored in the index information of the copied file. This way, only the data of one file needs to be stored on the storage medium, thus improving the utilization rate of the storage medium.

[0008] Based on the method provided in the first aspect, in one possible implementation, the first index node includes a first index block, which in turn includes blocks corresponding to multiple data items in the first file. The block corresponding to any data item is used to store index information indicating the corresponding data. In this scenario, the process of writing the identifier of the target data block as index information of the first data into the first index node is as follows: the identifier of the target data block is written into the block corresponding to the first data. Furthermore, the file system also includes a second index node corresponding to the second file. The second index node includes a second index block, which in turn includes blocks corresponding to multiple data items in the second file. In this scenario, in this method, first shared block information is written into the block corresponding to the first data. The first shared block information indicates the identifier of the block corresponding to the second data.

[0009] In this embodiment, since the same data can be stored in the same data block across different files, different files can share the same data block. To facilitate unified management of files sharing the same data block, the content of the file's inode can be expanded. The purpose of expanding the inode's content is to enable the identification of which files share the data block through the inode. The aforementioned shared block information is one type of expanded information.

[0010] Based on the method provided in the first aspect, in one possible implementation, the first shared block information includes the identifier of the second index block and the sequence number of the block in the second index block corresponding to the second data.

[0011] The first shared block information indicates the identifier of the block corresponding to the second data. Thus, by using the block corresponding to the first data, one can query relevant information about blocks in other files that contain the same data as the first data, improving the efficiency of querying inodes of other files using shared block information.

[0012] Based on the method provided in the first aspect, in one possible implementation, the method updates the second shared block information in the block corresponding to the second data in the second index block. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data.

[0013] Since the first data in the first file and the second data in the second file are the same data, if the first index node stores information indicating the block corresponding to the data in the second file that is the same as the first data, then the index node of the second file will also store information indicating the block corresponding to the data in other files that is the same as the second data. In this way, the information stored in the various blocks sharing the same data block can form a circular chain. This further facilitates the unified management of the shared data in the future.

[0014] Based on the method provided in the first aspect, in one possible implementation, a first shared quantity information is written into the block corresponding to the first data, the first shared quantity information indicating the number of files in the data that include the data in the target data block, excluding the first file.

[0015] The purpose of expanding this shared quantity information is to allow for quick determination of whether there are blocks in other files that point to the same data block when the file needs to be modified later. This makes it easier to determine whether other operations need to be performed and improves the convenience of subsequent data management.

[0016] Based on the method provided in the first aspect, in one possible implementation, the method updates the second shared quantity information in the block corresponding to the second data in the second index block, the second shared quantity information indicating the number of files in the data that include the data in the target data block, excluding the second file.

[0017] Similarly, based on the above circular chain configuration, after writing the first shared quantity information into the block corresponding to the first data, the second shared quantity information in the block corresponding to the second data in the second index block can also be updated. The second shared quantity information indicates the number of files in the data that include data from the target data block, excluding the second file. This forms a circular chain among the shared quantities, further facilitating the unified management of shared data in the future.

[0018] Based on the method provided in the first aspect, in one possible implementation, upon detecting a data modification instruction based on the first data, a data block configured for the modified first data is obtained; the modified first data is written into the configured data block; and the index information of the first data is updated based on the identifier of the configured data block.

[0019] After the first file is written to the file system using the method provided in this application embodiment, if the first data in the first file needs to be modified subsequently, the modified first data and the second data in the second file will no longer be the same. Therefore, it is necessary to rewrite the modified first data and modify the index information of the first data accordingly. This ensures that the index nodes of the first file can represent the actual storage location of the data in the first file in real time, thereby improving the accuracy of subsequent data queries.

[0020] Based on the method provided in the first aspect, in one possible implementation, the first index node includes a first index block, which includes blocks corresponding to multiple pieces of data in the first file. The block corresponding to any piece of data stores index information indicating the corresponding data. The file system also includes a second index node corresponding to the second file, which includes a second index block, which includes blocks corresponding to multiple pieces of data in the second file. In this scenario, in the method, the first shared block information in the block corresponding to the first data is cleared. The first shared block information indicates the identifier of the block corresponding to the second data.

[0021] Based on the above, it can be seen that shared block information can be extended into the inodes of each file. In this scenario, after writing the modified first data, since the modified first data and the second data in the second file will no longer be the same, it is also necessary to clear the first shared block information in the block corresponding to the first data. The first shared block information indicates the identifier of the block corresponding to the second data. This improves the timeliness of data stored in the file system.

[0022] Based on the method provided in the first aspect, in one possible implementation, the method updates the second shared block information in the block corresponding to the second data in the second index block. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data.

[0023] The new second shared block information will no longer include information about the block corresponding to the first data, thus indicating that the second data and the first data are no longer the same. This improves the timeliness of data stored in the file system.

[0024] Based on the method provided in the first aspect, in one possible implementation, the method clears the first shared quantity information in the block corresponding to the first data, the first shared quantity information indicating the number of files in the data that include the data in the target data block, excluding the first file.

[0025] Based on the above, it can be seen that the inodes of each file can be extended with shared quantity information. In this scenario, after writing the modified first data, since the modified first data and the second data in the second file will no longer be the same, it is also necessary to clear the first shared quantity information in the block corresponding to the first data. The first shared quantity information indicates the number of files, excluding the first file, that include the data in the target data block. This improves the timeliness of the data stored in the file system.

[0026] Based on the method provided in the first aspect, in one possible implementation, the method updates the second shared quantity information in the block corresponding to the second data in the second index block, the second shared quantity information indicating the number of files in the data that include the data in the target data block, excluding the second file.

[0027] The second shared quantity information in the block corresponding to the second data in the second index block is updated. This second shared quantity information indicates the number of files in the data that include data from the target data block, excluding the second file. This improves the timeliness of data stored in the file system.

[0028] Secondly, a data writing device is provided, which has the function of implementing the data writing method behavior described in the first aspect. The data writing device includes at least one module for implementing the data writing method provided in the first aspect.

[0029] Thirdly, a computer device is provided, comprising a processor and a memory. The memory stores a program that supports the computer device in executing the data writing method provided in the first aspect, and stores data related to implementing the data writing method provided in the first aspect. The processor is configured to execute the program stored in the memory. The operating means of the storage device may further include a communication bus for establishing a connection between the processor and the memory.

[0030] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the data writing method described in the first aspect.

[0031] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the data writing method described in the first aspect above.

[0032] The technical effects achieved by the second, third, fourth, and fifth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a file system architecture provided in an embodiment of this application;

[0034] Figure 2 This is a schematic diagram of an index tree in an index node provided in an embodiment of this application;

[0035] Figure 3 This is a flowchart of a data writing method provided in an embodiment of this application;

[0036] Figure 4 This is a schematic diagram illustrating the writing of data in a file copying scenario provided in an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of a data field in an extended block provided in an embodiment of this application;

[0038] Figure 6 This is a schematic diagram illustrating an update of a data field in an extended block, as provided in an embodiment of this application.

[0039] Figure 7 This is a schematic diagram of a file copying process provided in an embodiment of this application;

[0040] Figure 8 This is a schematic diagram of another data field in an updated extension block provided in an embodiment of this application;

[0041] Figure 9 This is a flowchart illustrating a process for modifying data in a copied file, provided in an embodiment of this application.

[0042] Figure 10 This is a block diagram of a data writing device provided in an embodiment of this application;

[0043] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0045] Before providing a detailed explanation of the embodiments of this application, the application scenarios of the embodiments of this application will be explained first.

[0046] The method provided in this application is applied to a scenario where data is written to a file system as a file. For ease of explanation later, the architecture of the file system will be explained first.

[0047] To facilitate the construction of a file system, each partition in a storage medium, such as a disk, is pre-divided into blocks. Each block corresponds to a contiguous segment of storage space in the storage medium, and the file system can be built based on the blocks on the storage medium. Figure 1 This is a schematic diagram of a file system architecture provided in an embodiment of this application. This file system corresponds to a partition on the storage medium. For example... Figure 1 As shown, the file system includes a boot block and various block groups. Figure 1 The various block groups are labeled as block group 0, block group 1, ..., block group N. The boot block stores information such as the operating system type of the partition, the partition's starting address, and the partition size as specified in the standard. Each partition on the disk contains only one boot block.

[0048] The information included in each block group can be as follows Figure 1 The information included in block group 0 is shown below. For example... Figure 1 As shown, block group 0 includes a super block, group descriptions, a datablock bitmap, an inode bitmap, an inode table, and data blocks.

[0049] The system consists of several parts: a superblock (occupying one block on the storage medium) and a data block (occupying one block on the storage medium). The superblock stores information about the file system, such as the mount time and the last time a file was written to the file system. A block group descriptor (occupying multiple blocks on the storage medium) stores descriptive information about each block group, such as the block number at the beginning of each group. A data block bitmap occupies one block on the storage medium. Each bit in the bitmap indicates whether a data block is used; 1 indicates the block is used, and 0 indicates it is not. An inode bitmap occupies one block on the storage medium. Each bit in the bitmap indicates whether an inode is used; 1 indicates it is used, and 0 indicates it is not. An inode table occupies multiple blocks on the storage medium and stores inodes. Each inode corresponds to a file and stores the location of the data in that file on the storage medium. In other words, the inode stores the index information of the data in the file, allowing data to be retrieved from the storage medium. Data blocks store the content of the file.

[0050] An inode contains multiple index blocks (i_block). To avoid the file size limit being too low due to the inode's own capacity limitation, index blocks can store index information of the data in the file. Index blocks can also store index information of other index blocks.

[0051] In this system, the index information of any data indicates the identifier of the data block storing the corresponding data. Based on the identifier of the data block, the specific address of the data block on the storage medium can be found, which is also the physical address of the data block, thereby allowing the data in the data block to be read. The identifier of the data block can be the logical block number of the data block or the location number of the data block on the storage medium such as a hard disk. This location number is also called the physical block number corresponding to the data block.

[0052] Furthermore, the index information of the aforementioned other index blocks can be specifically understood as: the ability to locate the index block based on this index information, and then using other index blocks to indicate the index information of the data. Therefore, an index block can store the index information of the data, and can also be used for the index information of other index blocks.

[0053] Furthermore, to facilitate the management of information stored in the index blocks, multiple extents (cxtent) and index blocks (extent-idx) are pre-stored within each index block. Each extent stores the index information of a single data entry, and each extent stores the index information of only one data entry or one other index block. Index blocks store the index information of another index block. Thus, the information stored within an index block can be distinguished by the extent and the index block (extent-idx). By introducing index blocks, the index information of each data entry in the file can be stored in the form of an index tree; that is, the index information in the index nodes is stored in the form of an index tree. Each node in the index tree represents an index block. Leaf nodes correspond to index blocks storing the index information of data, while other nodes in the index tree, excluding leaf nodes, correspond to index blocks storing the index information of other index blocks.

[0054] For ease of explanation later, here is a brief description of the data format of the index information stored in the extent and index block (extent-idx) of the file system.

[0055] The data structure of an extent:

[0056]

[0057] The first line of code above indicates the logical number of the first data block among the multiple consecutive data blocks pointed to by the block. The logical number refers to the sequence number of the data block after the content of the entire file is divided into blocks. The second line of code above indicates the number of the multiple consecutive data blocks pointed to by the block. The third and fourth lines of code above indicate the high 16 bits and low 32 bits of the physical block number corresponding to the multiple data blocks pointed to by the block, respectively.

[0058] Data structure of the index block (extent-idx):

[0059]

[0060] The first line of code above indicates the logical number of the index block pointed to by the index block. The block number, which is the combination of the second and third lines of code above, indicates the physical block number of the index block pointed to by the index block. This physical block number indicates the specific physical location of the index block on a storage medium such as a hard disk.

[0061] It should be noted that the two data formats described above are only illustrative examples using ext4. Furthermore, when the file system is ext4, the aforementioned blocks can also be represented as ext4_extent, and the aforementioned index blocks can be represented as ext4_extent_idx. The specific data formats in inodes of other file system types will not be illustrated further.

[0062] Figure 2 This is a schematic diagram of an index tree in an index node provided in an embodiment of this application. For example... Figure 2 As shown, index block 1 in the index node includes one index block, which points to index block 2. This means that the index block stores the index information for index block 2. Subsequent phrases like "pointing to index block" refer to this explanation. Index block 2 includes two index blocks, which point to index block 3 and index block 4 respectively. Index block 3 includes multiple blocks, each pointing to multiple consecutive data blocks. Figure 2 The first block in index block 3 points to multiple data blocks numbered 1. These consecutive data blocks are used to store a single piece of data; that is, each block stores the index information for one piece of data. Index block 4 also includes multiple blocks, each pointing to multiple consecutive data blocks. Figure 2 The first block in index block 4 points to multiple data blocks numbered 2.

[0063] It should be noted that, Figure 2 The index tree shown is for illustrative purposes only and does not constitute a limitation on the specific structure of the index tree provided in the embodiments of this application.

[0064] The method provided in this application embodiment is applied to the aforementioned file system. It should be noted that the file system in this application embodiment can be a fourth-generation extended file system (ext4), or other types of file systems, which will not be described in detail here.

[0065] Currently, with the development of information and network technologies, data volume is experiencing explosive growth. The increasing demand for data storage has intensified the conflict between data storage needs and storage media costs, making data reduction technologies particularly important. For example, after storing data through the aforementioned file system, the storage medium often contains a large amount of duplicate data, resulting in significant waste of storage media. For instance, due to file version updates, multiple file copies are stored on the file system, and these copies often contain the same data, further wasting storage media. Based on this technical problem, the method provided in this application aims to provide a method for writing data that can improve the utilization of storage media.

[0066] The data writing method provided in the embodiments of this application will be explained in detail below.

[0067] Figure 3 This is a flowchart illustrating a method for writing data according to an embodiment of this application. This method is applied to a computer device equipped with a file system. Figure 3 As shown, the method includes the following steps.

[0068] Step 301: The computer device obtains a first file to be written. The first file includes first data. The first data is the same as the second data in the second file stored in the file system. The second data is stored in the target data block of the file system. The file system also includes a first index node corresponding to the first file. The first index node is used to store the index information of each piece of data included in the first file. The index information of any piece of data indicates the identifier of the data block storing the corresponding data.

[0069] In one possible implementation, step 301 can be applied to scenarios involving file copying. For example, in a scenario where files need to be copied during file version updates. In this case, the first file to be written is the second file already stored in the file system. In this scenario, the first data can be any data from the first file.

[0070] In another possible implementation, step 301 can also be applied to the scenario of writing a new file. In this case, for the first file to be written, all files already stored in the file system are traversed, and then it is checked whether there is data in the file system that is identical to a certain piece of data in the first file. If identical data exists, the identical data in the first file is identified as the first data, the identical data stored in the file system is referred to as the second data, and the file containing the second data is referred to as the second file.

[0071] It should be noted that the above two scenarios are merely illustrative and do not constitute a limitation on the application scenarios of the data writing method provided in this application. The data writing method provided in this application can be applied to any scenario where data already exists in the storage file system.

[0072] After determining the first data through any of the above scenarios, since the first data and the second data are the same data, and the second data has already been stored in the target data block in the file system, in this embodiment of the application, it is not necessary to store the first data repeatedly to avoid wasting storage media. The index information of the second data can be used as the index information of the first data through step 302 below. In this way, the target data block can be queried using the index information of the first data, thereby obtaining the data in the target data block, that is, reading the first data.

[0073] Step 302: The computer device writes the identifier of the target data block as the index information of the first data into the first index node.

[0074] In one possible implementation, the first index node includes a first index block, which contains blocks corresponding to multiple data items in the first file. The block corresponding to any data item is used to store index information indicating that data. In this case, the process of writing the identifier of the target data block as index information of the first data into the first index node can be: writing the identifier of the target data block into the block corresponding to the first data in the first index block.

[0075] It should be noted that a first index node has already been configured for the first file before it is written, and each piece of data in the first file has a corresponding index block and region in the first index node. Therefore, when writing the index information of the first data, the index information of the first data can be directly written into the corresponding region in the first index block corresponding to the first data. As for whether the index blocks in the first index node are stored in the form of an index tree, this embodiment of the application does not limit this.

[0076] Furthermore, in one possible implementation, the identifier of the aforementioned target data block may include the information indicated by the four lines of code in the data structure of the aforementioned extent. That is, the identifier of the target data block can be indicated by four aspects of information. These four aspects include: the logical number of the first data block among multiple consecutive data blocks storing the first data; the number of these multiple consecutive data blocks; and the high 16 bits and low 32 bits of the physical block number corresponding to these multiple data blocks.

[0077] It should be noted that the above example of writing the identifier of the target data block into the block corresponding to the first data in the first index block is only for ext4. Optionally, in other types of file systems, the identifier of the target data block can also be written into the corresponding position of the index information used to store the first data, based on the data organization method in the inode of other file systems. Examples will not be given here.

[0078] Furthermore, in the scenario described above where the second file is copied to obtain the first file, the index information of each data in the first file can be determined by referring to the method for determining the index information of the first data, thereby enabling the index information of the entire file in the second file to be stored in the index node of the first file.

[0079] Figure 4 This is a schematic diagram illustrating the writing of data in a file copying scenario provided by an embodiment of this application. Currently, it is necessary to copy a second file and store it again in the file system as the first file. For example... Figure 4As shown, after copying the entire index information of the second file to the inode of the first file, the data blocks ultimately pointed to by each index block in the inode of the first file are the data blocks storing the data in the second file. That is, the data block pointed to by the leaf nodes of the index tree of the first file and the data block pointed to by the leaf nodes of the index tree of the second file are the same data block. For example, Figure 4 The first block of index block aaa in the index node of the second file and the first block of index block ccc in the index node of the first file both point to the data block with the label 1. Figure 4 The first block in index block bbb of the inode of the second file and the first block in index block ddd of the inode of the first file both point to the data block numbered 2. In this way, it is not necessary to re-store all the data in the first file in the file system, thus saving storage space on the computer device and improving the utilization of the storage medium.

[0080] In this embodiment, since the same data can be stored in the same data block in different files, different files can share the same data block. To facilitate unified management of files sharing the same data block, the content of the file's inode can be expanded. The purpose of expanding the inode's content is to enable the identification of which files share the data block through the inode. The expanded content of this embodiment is explained in detail below.

[0081] (1) Extend shared block information in the block of index information of the data stored in the file. The shared block information indicates the identifier of the block corresponding to other data that shares the same data block with the data block pointed to by the block.

[0082] In this scenario, since the file system also includes a second inode corresponding to the second file, and the second inode includes a second index block, which in turn includes blocks corresponding to multiple pieces of data in the second file, after writing the index information of the first data to the block corresponding to the first data in the first index block, first shared block information can also be written to the block corresponding to the first data in the first index block. The first shared block information indicates the identifier of the block corresponding to the second data. Thus, by using the block corresponding to the first data, relevant information about the blocks corresponding to data in other files that have the same first data can be retrieved.

[0083] In one possible approach, the identifier of the block corresponding to the second data can be indicated by the identifier of the second index block and the sequence number of the block corresponding to the second data within the second index block. That is, the aforementioned first shared block information includes the identifier of the second index block and the sequence number of the block corresponding to the second data within the second index block.

[0084] In the above implementation, when the file system is ext4, it is equivalent to extending two data fields within the ext4 block (ext4_extent). These two data fields are labeled ee_sm_block and ee_index, respectively. ee_sm_block stores the index block number (this index block also contains ext4_extents pointing to the same data block). ee_index stores which ext4_extent in the index block indicated by ee_sm_block points to the same data block. For example, in steps 301 and 302 above, ee_sm_block stores the identifier of the second index block, and ee_index stores the sequence number of the block corresponding to the second data within the second index block.

[0085] Optionally, the identifier of the block corresponding to the second data can also be the physical address of the block on the storage medium, such as the block number of the physical block corresponding to the block. This application does not specifically limit this aspect.

[0086] Since the first data in the first file and the second data in the second file are the same data, if the first index node stores information indicating the block corresponding to the data in the second file that is the same as the first data, then the index node of the second file also stores information indicating the block corresponding to the data in other files that is the same as the second data. In this way, the information stored in the various blocks that share the same data block can form a circular chain.

[0087] Based on the above scenario, after writing the first shared block information into the block corresponding to the first data in the first index block, the second shared block information in the block corresponding to the second data in the second index block can also be updated. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data. Assuming that the other data only includes the aforementioned first data, the second shared block information indicates the identifier of the block corresponding to the aforementioned first data. The identifier of the block corresponding to the first data will not be repeated here.

[0088] (2) Expand the shared quantity information in the block containing the index information of the stored data. The shared quantity information indicates the number of files in the data that include the data in the data block pointed to by the block, excluding the current file.

[0089] The purpose of expanding this shared quantity information is that when the file needs to be modified later, it is possible to quickly determine whether there are blocks in other files and the blocks that point to the same data blocks, so as to determine whether other operations need to be performed. The specific details will be explained in detail in the subsequent embodiments of modifying the first file, and will not be elaborated here.

[0090] At this point, after writing the index information of the first data into the block corresponding to the first data, the first shared quantity information can also be written into the block corresponding to the first data. The first shared quantity information indicates the number of files in the data that include the data in the target data block, excluding the first file.

[0091] In the above implementation, when the file system is ext4, it is equivalent to extending another data field into the ext4 block (ext4_extent), which can be marked as ee_count. Here, ee_count represents the shared data, ee_count indicates that ee_count+1 files share the data block, and ee_count is 0, indicating that the current file exclusively owns the data block.

[0092] Similarly, based on the above circular chain configuration, after writing the first shared quantity information into the block corresponding to the first data, the second shared quantity information in the block corresponding to the second data in the second index block can also be updated. The second shared quantity information indicates the number of files in the data that include data from the target data block, excluding the second file. This forms a circular chain among the shared quantities as well.

[0093] It should be noted that the content (1) and content (2) extended in the index node can be configured simultaneously or only one of them can be configured. This application embodiment does not limit this. When only the content (1) is extended, the number of shares can be parsed based on the extended shared block information, which will not be described in detail here. When only the content (2) is extended, the shared block information can be queried by traversing the index nodes of each file, which will also not be described in detail here.

[0094] When the contents (1) and (2) of the inode are configured simultaneously, if the file system is ext4, it is equivalent to extending three data fields in the ext4 block (ext4_extent). These three data fields are ee_sm_block, ee_index, and ee_count, as mentioned above.

[0095] At this point, the data structure of the expanded block (ext4_extent) is as follows:

[0096]

[0097] In the four bolded lines of code above, the second and third lines represent the high and low bits of the index block number in ee_sm_block. The other lines of code, excluding the bolded ones, have already been explained and will not be repeated here.

[0098] Assuming that both shared block information and shared quantity information are expanded, the following example of copying data from the second file to the first file will be used to illustrate the above process in more detail.

[0099] When writing to the second file, the three extended data fields mentioned above are set to 0 in the extent pointing to each data block, indicating that no other file is currently sharing that data block. For example... Figure 5 As shown, in the index node of the second file, any extent pointing to a data block includes not only the relevant information of the data block (specifically, the index information of the stored data, which has been explained in the data structure of the aforementioned extent), but also three data fields: “block: 0”, “index: 0”, and “count: 0”, indicating that there is currently no data in other files that is consistent with the data in the data block pointed to by this extent. Figure 5 The "0, 0, 0" in the text correspond to the aforementioned "block: 0", "index: 0", and "count: 0" respectively. Figure 5 The abbreviation is simply for the sake of convenience.

[0100] When copying data from the second file to the first file, in addition to copying all the index information from the second index node of the second file to the first index node of the first file, the information in the three extended data fields of the block in the first index node and the three extended data fields of the block in the second index node are also updated respectively. For example... Figure 6 As shown, the first block in the index block 'aaa' of the second file and the first block in the index block 'ccc' of the second file point to the same data block. Therefore, the "block:0", "index:0", and "count:0" in the first block of the index block 'aaa' of the second file are updated to "block:ccc", "index:1", and "count:1". The "block:0", "index:0", and "count:0" in the first block of the index block 'ccc' of the first file are updated to "block:aaa", "index:1", and "count:1".

[0101] In addition, such as Figure 6 As shown, the first block in the index block bbb of the second file and the first block in the index block ddd of the second file point to the same data block. Therefore, the "block:0", "index:0", "count:0" in the first block of the index block bbb of the second file is updated to "block:ddd", "index:1", "count:1". The "block:0", "index:0", "count:0" in the first block of the index block ddd of the first file is updated to "block:bbb", "index:1", "count:1".

[0102] At this time, as Figure 6 As shown by the connecting line at the midpoint, these three data fields form a circular chain in the index tree.

[0103] The above content can also be accessed through Figure 7 The flowchart shown is used to illustrate this. Regarding... Figure 7 The details in this document will not be explained in detail here.

[0104] It should be noted that the above embodiment is illustrated by copying data from the second file to the first file. Optionally, if only part of the data in the first file to be written is the same as the data in the second file, the same data can be written in the same way as described above. For different data, unused data blocks can be allocated for writing; the specific implementation method will not be described in detail here.

[0105] In addition, through Figure 3 In the illustrated embodiment, after the first file is written to the file system, if the first data in the first file needs to be modified subsequently, the modified first data and the second data in the second file will no longer be the same. Therefore, it is necessary to rewrite the modified first data and modify the index information of the first data accordingly.

[0106] In one possible implementation, the process of rewriting the modified first data and modifying the index information of the first data accordingly is as follows: when a data modification instruction based on the first data is detected, a data block configured for the modified first data is obtained, the modified first data is written into the configured data block, and the index information of the first data is updated based on the identifier of the configured data block.

[0107] The aforementioned update of the index information of the first data based on the identifier of the configured data block means replacing the identifier of the original target data block with the identifier of the configured data block as the index information of the first data. Thus, subsequent searches based on the index information of the first data will retrieve the modified first data.

[0108] Furthermore, based on the above, it is known that shared block information can be extended into the inodes of each file. In this scenario, after writing the modified first data, since the modified first data and the second data in the second file will no longer be the same, it is also necessary to clear the first shared block information in the block corresponding to the first data. The first shared block information indicates the identifier of the block corresponding to the second data.

[0109] In addition, the existing shared block information in the second file needs to be updated. Specifically, the second shared block information in the block corresponding to the second data in the second index block needs to be updated. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data. The updated second shared block information will no longer include information related to the block corresponding to the first data, thereby indicating that the second data and the first data are no longer the same data.

[0110] Furthermore, based on the above, it is known that the inodes of each file can also be extended with shared quantity information. In this scenario, after writing the modified first data, since the modified first data and the second data in the second file will no longer be the same, it is also necessary to clear the first shared quantity information in the block corresponding to the first data. The first shared quantity information indicates the number of files, excluding the first file, that include the data in the target data block.

[0111] In addition, the existing shared quantity information in the second file needs to be updated. Specifically, the second shared quantity information in the block corresponding to the second data in the second index block needs to be updated. The second shared quantity information indicates the number of files in the data other than the second file that include the data in the target data block.

[0112] It should be noted that the above-mentioned updates to shared block information or shared quantity information are all based on the content extended in the aforementioned blocks, and will not be explained in detail here.

[0113] Assuming that both shared block information and shared quantity information are expanded, the following will be done by modifying... Figure 6 The above process will be explained in detail using the data corresponding to the first block in the index block ccc in the first file as an example.

[0114] like Figure 8 As shown, the modified first data is stored in the reallocated data block. Figure 8 In the data block marked 3. At this time, as Figure 8 As shown, the first block in the index block ccc will no longer point to the original data block. Figure 8 Instead of pointing to the data block marked as 1, it points to the reassigned data block. Figure 8The data block marked as 3 in the middle represents the modified first data stored in the reallocated data block.

[0115] At this point, the above-mentioned clearing of the first shared block information and the first shared quantity information specifically refers to updating "block:aaa", "index:1", and "count:1" in the first block of the index block ccc of the first file to "block:0", "index:0", and "count:0".

[0116] The above-mentioned update of the second shared block information and the second shared quantity information refers to updating "block: ccc", "index: 1", and "count: 1" in the first block of the index block aaa of the second file to "block: 0", "index: 0", and "count: 0".

[0117] At this time, as Figure 8 As shown, relative to Figure 6 The circular chain between the first block in index block aaa and the first block in index block ccc. Figure 8 There is no longer a circular chain between the first block in index block aaa and the first block in index block ccc.

[0118] Furthermore, during the modification of the first file, to quickly determine which data in the first file shares the same data blocks with data in other files, and thus to determine whether the aforementioned clearing and updating of shared block information and sharing quantity information is necessary, for any data in the first file, first determine whether the sharing quantity information in the corresponding block is greater than 0. If it is greater than 0, then perform the aforementioned clearing and updating operations. If the sharing quantity is equal to 0, then directly write the modified data after the newly allocated data block.

[0119] The above content can also be accessed through Figure 9 The flowchart shown is used to illustrate this. Regarding... Figure 9 The details in this document will not be explained in detail here.

[0120] In summary, in this embodiment, since the first data and the second data are identical, and the second data has already been stored in the target data block of the file system, it is unnecessary to repeatedly store the first data. The identifier of the target data block can be used as the index information of the first data. Subsequently, the target data block can be queried using the index information of the first data, thereby retrieving the data within the target data block, i.e., reading the first data. This avoids wasting storage media. Furthermore, the identifier of the target data block can indicate its specific location on the storage media; that is, the identifier of the target data block can serve as the index information for the data stored within it. Therefore, in file copying scenarios, it is unnecessary to re-store all the data of the file into the file system; only the index information of the original file needs to be stored in the index information of the copied file. Thus, only the data of one file needs to be stored on the storage medium, thereby improving the utilization rate of the storage medium.

[0121] Figure 10 This is a schematic diagram of a data writing device provided in an embodiment of this application. The device is deployed in a computer device, and the computer device has a file system mounted on it. Figure 10 As shown, the device 1000 includes the following modules.

[0122] Module 1001 is used to retrieve the first file to be written. For details on its implementation, please refer to [reference needed]. Figure 3 Step 301 in the embodiment.

[0123] The device first file includes first data, and the device first data and the second data in the second file stored in the device file system are the same data. The device second data is stored in the target data block of the device file system. The device file system also includes a first index node corresponding to the device first file. The device first index node is used to store the index information of each data included in the device first file. The index information of any data indicates the identifier of the data block storing the corresponding data.

[0124] The write module 1002 is used to write the identifier of the target data block of the device as the index information of the first data of the device into the first index node of the device. For specific implementation details, please refer to [reference needed]. Figure 3 Step 302 in the embodiment.

[0125] Optionally, the first index node of the device includes a first index block, which includes blocks corresponding to multiple data in the first file of the device, wherein the block corresponding to any data is used to store index information indicating the corresponding data; the writing module of the device is used to:

[0126] Write the identifier of the target data block of the device into the block corresponding to the first data of the device;

[0127] The device file system also includes a second index node corresponding to the second file of the device. This second index node includes a second index block, and each second index block includes blocks corresponding to multiple data items in the second file of the device. The device write module is further configured to:

[0128] First shared block information is written into the block corresponding to the first data of the device, and the first shared block information of the device indicates the identifier of the block corresponding to the second data of the device.

[0129] Optionally, the first shared block information of the device includes the identifier of the second index block of the device and the sequence number of the block in the second index block of the device that corresponds to the second data of the device.

[0130] Optionally, the device further includes:

[0131] The update module is used to update the second shared block information in the block corresponding to the second data of the device in the second index block of the device. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data of the device.

[0132] Optionally, the device's writing module is also used for:

[0133] First shared quantity information is written into the block corresponding to the first data of the device. The first shared quantity information of the device indicates the number of files in the data that include the data in the target data block of the device, in addition to the first file of the device.

[0134] Optionally, the device further includes:

[0135] The update module is used to update the second shared quantity information in the block corresponding to the second data of the device in the second index block of the device. The second shared quantity information indicates the number of files in the data other than the second file of the device that include the data in the target data block of the device.

[0136] Optionally, it is characterized in that,

[0137] The device acquisition module is also used to acquire a data block configured for the modified first data when a data modification instruction based on the first data of the device is detected;

[0138] The device writing module is also used to write the modified first data of the device into the configured data block;

[0139] The device also includes an update module for updating the index information of the device's first data based on the identifier of the data block configured in the device.

[0140] Optionally, the first index node of the device includes a first index block, and the first index block of the device includes blocks corresponding to multiple data in the first file of the device, and the block corresponding to any data is used to store index information indicating the corresponding data. The file system of the device also includes a second index node corresponding to the second file of the device, and the second index node of the device includes a second index block, and the second index block of the device includes blocks corresponding to multiple data in the second file of the device.

[0141] The device also includes:

[0142] The clearing module is used to clear the first shared block information in the block corresponding to the first data of the device, the first shared block information indicating the identifier of the block corresponding to the second data of the device.

[0143] Optionally, the device update module is also used for:

[0144] Update the second shared block information in the block corresponding to the second data of the device in the second index block of the device. The second shared block information of the device indicates the identifier of the block corresponding to other data that is the same as the second data of the device.

[0145] Optionally, the device further includes:

[0146] Clear the first shared quantity information in the block corresponding to the first data of the device, the first shared quantity information of the device indicating the number of files in the data other than the first file of the device that include the data in the target data block of the device.

[0147] Optionally, the cleaning module of the device is also used for:

[0148] Update the second shared quantity information in the block corresponding to the second data of the device in the second index block of the device. The second shared quantity information of the device indicates the number of files in the data that include the data in the target data block of the device, in addition to the second file of the device.

[0149] In summary, in this embodiment, since the first data and the second data are identical, and the second data has already been stored in the target data block of the file system, it is unnecessary to repeatedly store the first data. The identifier of the target data block can be used as the index information of the first data. Subsequently, the target data block can be queried using the index information of the first data, thereby retrieving the data within the target data block, i.e., reading the first data. This avoids wasting storage media. Furthermore, the identifier of the target data block can indicate its specific location on the storage media; that is, the identifier of the target data block can serve as the index information for the data stored within it. Therefore, in file copying scenarios, it is unnecessary to re-store all the data of the file into the file system; only the index information of the original file needs to be stored in the index information of the copied file. Thus, only the data of one file needs to be stored on the storage medium, thereby improving the utilization rate of the storage medium.

[0150] It should be noted that the data writing device provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data writing device and the data writing method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0151] Figure 11 This is a schematic diagram of the structure of a computer device provided in the embodiments of this application. The file system in the foregoing embodiments can be mounted on... Figure 11 On the computer device shown, thus enabling Figure 11 The computer device shown implements the data writing method provided in the embodiments of this application. See also Figure 11 The computer device includes at least one processor 1101, a communication bus 1102, a memory 1103, and at least one communication interface 1104.

[0152] The processor 1101 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0153] The communication bus 1102 may include a path for transmitting information between the aforementioned components.

[0154] The memory 1103 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 1103 may exist independently and be connected to the processor 1101 via the communication bus 1102. The memory 1103 may also be integrated with the processor 1101.

[0155] The memory 1103 stores program code for executing the scheme of this application, and its execution is controlled by the processor 1101. The processor 1101 executes the program code stored in the memory 1103. The program code may include one or more software modules.

[0156] Communication interface 1104 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0157] In a specific implementation, as one example, a computer device may include multiple processors, for example... Figure 11 The processors 1101 and 1105 shown are illustrated. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0158] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0159] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0160] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0161] The above descriptions are embodiments provided in this application and are not intended to limit the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for writing data, characterized in that, Applied to a computer device, wherein a file system is mounted on the computer device, the method includes: Get the first file to be written; The first file includes first data, which is the same as the second data in the second file stored in the file system. The second data is stored in the target data block of the file system. The file system also includes a first index node corresponding to the first file. The first index node is used to store index information of each piece of data included in the first file. The index information of any piece of data indicates the identifier of the data block storing the corresponding data. The first index node includes a first index block. The first index block includes blocks corresponding to multiple pieces of data in the first file. The block corresponding to any piece of data is used to store index information indicating the corresponding data. The identifier of the target data block is used as the index information of the first data and written into the block corresponding to the first data; Write first shared block information into the block corresponding to the first data. The first shared block information indicates the identifier of the block corresponding to the second data. The first shared block information includes the identifier of the second index block and the sequence number of the block corresponding to the second data in the second index block. Update the second shared block information in the block corresponding to the second data in the second index block. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data.

2. The method as described in claim 1, characterized in that, The file system also includes a second index node corresponding to the second file, the second index node includes a second index block, and the second index block includes blocks corresponding to multiple data items in the second file.

3. The method as described in any one of claims 1 to 2, characterized in that, The method further includes: Write first shared quantity information into the block corresponding to the first data. The first shared quantity information indicates the number of files in the data that include the data in the target data block, excluding the first file.

4. The method as described in claim 3, characterized in that, The method further includes: Update the second shared quantity information in the block corresponding to the second data in the second index block. The second shared quantity information indicates the number of files in the data other than the second file that include the data in the target data block.

5. The method as described in any one of claims 1 to 2 and 4, characterized in that, The method further includes: Upon detecting a data modification instruction based on the first data, obtain the data block configured for the modified first data; Write the modified first data into the configured data block; The index information of the first data is updated based on the identifier of the configured data block.

6. The method as described in claim 5, characterized in that, The first index node includes a first index block, which includes blocks corresponding to multiple data in the first file. The block corresponding to any data is used to store index information indicating the corresponding data. The file system also includes a second index node corresponding to the second file, which includes a second index block, which includes blocks corresponding to multiple data in the second file. The method further includes: Clear the first shared block information in the block corresponding to the first data, whereby the first shared block information indicates the identifier of the block corresponding to the second data.

7. The method as described in claim 6, characterized in that, The method further includes: Update the second shared block information in the block corresponding to the second data in the second index block. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data.

8. The method as described in any one of claims 6 or 7, characterized in that, The method further includes: Clear the first shared quantity information in the block corresponding to the first data, the first shared quantity information indicating the number of files in the data that include the data in the target data block, excluding the first file.

9. The method as described in claim 8, characterized in that, The method further includes: Update the second shared quantity information in the block corresponding to the second data in the second index block. The second shared quantity information indicates the number of files in the data other than the second file that include the data in the target data block.

10. A data writing device, characterized in that, The device is deployed on a computer device, the computer device having a file system mounted thereon, and the device includes: The acquisition module is used to acquire the first file to be written; The first file includes first data, which is the same as the second data in the second file stored in the file system. The second data is stored in the target data block of the file system. The file system also includes a first index node corresponding to the first file. The first index node is used to store index information of each piece of data included in the first file. The index information of any piece of data indicates the identifier of the data block storing the corresponding data. The first index node includes a first index block. The first index block includes blocks corresponding to multiple pieces of data in the first file. The block corresponding to any piece of data is used to store index information indicating the corresponding data. The writing module is used to write the identifier of the target data block as the index information of the first data into the block corresponding to the first data; Write first shared block information into the block corresponding to the first data. The first shared block information indicates the identifier of the block corresponding to the second data. The first shared block information includes the identifier of the second index block and the sequence number of the block corresponding to the second data in the second index block. An update module is used to update the second shared block information in the block corresponding to the second data in the second index block. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data.

11. The apparatus as claimed in claim 10, characterized in that, The first index node includes a first index block, which includes blocks corresponding to multiple data items in the first file. The block corresponding to any data item is used to store index information indicating the corresponding data. The writing module is used to: Write the identifier of the target data block into the block corresponding to the first data; The file system further includes a second index node corresponding to the second file. The second index node includes a second index block, and the second index block includes blocks corresponding to multiple data items in the second file. The writing module is specifically used for: Write the first shared block information into the block corresponding to the first data.

12. The apparatus as claimed in any one of claims 10 to 11, characterized in that, The writing module is also used for: Write first shared quantity information into the block corresponding to the first data. The first shared quantity information indicates the number of files in the data that include the data in the target data block, excluding the first file.

13. The apparatus as claimed in claim 12, characterized in that, The device further includes: An update module is used to update the second shared quantity information in the block corresponding to the second data in the second index block. The second shared quantity information indicates the number of files in the data other than the second file that include the data in the target data block.

14. The apparatus as described in any one of claims 10 to 11, 13, characterized in that, The acquisition module is further configured to acquire the data block configured for the modified first data when a data modification instruction based on the first data is detected. The writing module is also used to write the modified first data into the configured data block; The device further includes an update module for updating the index information of the first data based on the identifier of the configured data block.

15. The apparatus as claimed in claim 14, characterized in that, The first index node includes a first index block, which includes blocks corresponding to multiple data in the first file. The block corresponding to any data is used to store index information indicating the corresponding data. The file system also includes a second index node corresponding to the second file, which includes a second index block, which includes blocks corresponding to multiple data in the second file. The device further includes: The clearing module is used to clear the first shared block information in the block corresponding to the first data, wherein the first shared block information indicates the identifier of the block corresponding to the second data.

16. The apparatus as claimed in claim 15, characterized in that, The update module is also used for: Update the second shared block information in the block corresponding to the second data in the second index block. The second shared block information indicates the identifier of the block corresponding to other data that is the same as the second data.

17. The apparatus as claimed in any one of claims 15 or 16, characterized in that, The device further includes: Clear the first shared quantity information in the block corresponding to the first data, the first shared quantity information indicating the number of files in the data that include the data in the target data block, excluding the first file.

18. The apparatus as claimed in claim 17, characterized in that, The clearing module is also used for: Update the second shared quantity information in the block corresponding to the second data in the second index block. The second shared quantity information indicates the number of files in the data other than the second file that include the data in the target data block.

19. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store programs that enable the computer device to execute the method according to any one of claims 1-9, and to store data related to implementing the method according to any one of claims 1-9; The processor is configured to execute programs stored in the memory.

20. A computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1-9.

Images