File processing method and electronic device

Abstract

The application provides a file processing method and an electronic device, which can be applied to the technical field of data storage. The method comprises the following steps: in the process of writing a target file into a first storage device, slicing the target file according to a preset slice size to obtain at least one slice file; numbering the at least one slice file according to a slice number recorded in file metadata, and recording the slice number and an initial access heat value of each slice file in the file metadata; periodically calculating the access heat value of any slice file in the first storage device according to a preset time interval; and updating the access heat value of any slice file in the file metadata by using the calculated access heat value to obtain an updated access heat value of any slice file.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, and particularly relates to a file processing method and an electronic device. BACKGROUND

[0002] With the rapid development of cloud computing and big data technology, the amount of data stored in a distributed storage system is rapidly increasing, which may cause waste of high-performance storage medium resources and high cost of storing low-frequency access data. Therefore, cold and hot data grading technology stores hot data in a high-performance storage medium to guarantee business response speed, and migrates cold data to a low-cost medium to reduce storage costs.

[0003] However, the data grading in the related art usually takes an entire file as the minimum migration unit, which may cause waste of storage resources, system resources and bandwidth, and even affect read-write speed. Moreover, the data grading in the related art only supports data grading in a single storage device, lacks extension capability across devices and networks, and has insufficient data disaster recovery reliability. SUMMARY

[0004] In view of the above problems, the present application provides a file processing method and an electronic device.

[0005] According to a first aspect of the present application, a file processing method is provided, including: in a process of writing a target file into a first storage device, slicing the target file according to a preset slice size to obtain at least one slice file; numbering the at least one slice file according to a slice number recorded in file metadata, and recording a slice number and an initial access hotness value of each slice file in the at least one slice file into the file metadata; periodically calculating an access hotness value of any slice file in the first storage device according to a preset time interval; updating the access hotness value of the any slice file in the file metadata by using the calculated access hotness value to obtain an updated access hotness value of the any slice file, wherein the access hotness value is used to determine whether the any slice file is cold data.

[0006] The second aspect of the present application provides a file processing device, comprising a slicing module, a numbering module, a calculating module and an updating module. The slicing module is configured to slice a target file into at least one slice file according to a preset slice size during writing the target file into a first storage device. The numbering module is configured to number the at least one slice file according to a slice number recorded in file metadata, and record the slice number and an initial access heat value of each slice file in the file metadata. The calculating module is configured to periodically calculate an access heat value of any slice file in the first storage device according to a preset time interval. The updating module is configured to update the access heat value of the any slice file in the file metadata by using the calculated access heat value, and obtain an updated access heat value of the any slice file.

[0007] The third aspect of the present application provides an electronic device, comprising one or more processors, and a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0008] The fourth aspect of the present application further provides a computer readable storage medium having stored thereon a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method.

[0009] The fifth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method.

[0010] According to the file processing method provided by the present application, the target file written into the first storage device is dynamically divided into a plurality of slice files, and the access heat value is calculated in units of slices, thereby realizing finer-grained data management, i.e., judging whether the slice file is hot data or cold data in units of slices, so that the slice file can be used as the minimum migration unit in the subsequent process, and the waste of storage resources, system resources and bandwidth is avoided to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 An application scenario diagram of the file processing method according to an embodiment of the present application is shown;

[0012] Figure 2 A flowchart of the file processing method according to an embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram of cold data migration according to an embodiment of the present application is shown;

[0014] Figure 4 A structural block diagram of a file processing apparatus according to an embodiment of the present application is shown;

[0015] Figure 5 A block diagram of an electronic device adapted to implement a file processing method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0016] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely exemplary and is not intended to limit the scope of the application. In the following detailed description of embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that one or more embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0017] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present application. The terms "include" and "have" and the like used herein indicate the presence of the described features, steps, operations, and / or components but do not preclude the presence or addition of one or more other features, steps, operations, or components.

[0018] All terms used herein, including technical and scientific terms, have the same meanings as those generally understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present description, and should not be interpreted in an idealized or excessively formal manner.

[0019] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include at least one of the items enumerated, but not limited to the items enumerated (e.g., "a system having at least one of A, B, and C" should include a system having A alone, a system having B alone, a system having C alone, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc.).

[0020] In the technical solutions of the present application, the user information (including but not limited to user personal information, user image information, user equipment information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, application, and application of related data comply with relevant laws, regulations, and standards, take necessary security measures, do not violate public order and good customs, and provide corresponding operation portal for user selection authorization or refusal.

[0021] In the scenario of making automated decisions by using personal information, the method, device and system provided by the embodiments of the present application all provide corresponding operation entrances for the user to select to agree or reject the automated decision result; if the user selects to reject, the expert decision process is entered. The expression "automated decision" here refers to the activity of automatically analyzing, evaluating the behavior habits, interests and hobbies or economic, health and credit conditions of a person by a computer program and making decisions. The expression "expert decision" here refers to the activity of making decisions by a person who is engaged in a certain field of work, has special experience, knowledge and skills and reaches a certain professional level.

[0022] In the process of implementing the present application, it is found that only a small part of hot data in the storage resource needs high-frequency access, and the remaining cold data is in a low-frequency access or silent state for a long time, but occupies a large amount of expensive storage resources. Therefore, the cold and hot data grading technology is usually used to finely classify and manage data, and hot data is stored in high-performance storage media (such as SSD, Solid State Drive, solid state disk), and cold data is migrated to low-cost media (such as HDD, Hard Disk Drive, hard disk drive).

[0023] In the related art, data grading usually takes the entire file as the minimum migration unit, and according to factors such as file access frequency and access time, the file data that meets the migration condition is migrated to the low-cost storage medium in whole, and the data is deleted from the high-performance storage medium. When the file needs to be accessed again, the entire file is retrieved from the low-cost storage medium.

[0024] Based on this, the related art has the following problems: The strategy of whole file migration is unreasonable, when a small amount of data in a large file is frequently accessed, the file is still determined as hot data, and the entire file needs to be retained on the high-performance storage medium, thereby causing waste of storage resources. When part of the data in the cold data is accessed, the data of the entire file needs to be retrieved from the cold data, if the file capacity is large, it will cause waste of system resources and bandwidth, and even affect the read-write rate. Moreover, only two storage levels of a single storage device are supported, that is, only the HDD storage pool and the SSD storage pool of a single storage device can be graded and stored, and it cannot be extended to multi-device cross-network off-site storage, so that the data disaster recovery reliability is insufficient.

[0025] Therefore, the embodiments of the present application provide a file processing method, which dynamically divides a file written into a first storage device into a plurality of slice files, and calculates the access heat value in units of slices, thereby realizing finer-grained data grading management and avoiding resource waste to a certain extent.

[0026] Figure 1 An application scenario diagram of the file processing method according to the embodiments of the present application is shown.

[0027] As shown in Figure 1 The application scenario 100 according to this embodiment can include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or fiber optic cables, and the like.

[0028] A user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, and the like. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, and the like (only as examples).

[0029] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and support for web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, and the like.

[0030] For example, a user can send a data write request through the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0031] The server 105 can be a server that provides various services, such as a background management server that provides support for websites browsed by a user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can analyze and process received user requests and the like, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests, and the like) to the terminal device.

[0032] For example, the server 105 can write data of the target file into the first storage device in response to the received data write request. The server 105 can slice the target file according to a preset slice size to obtain at least one slice file during writing the target file into the first storage device, number the at least one slice file according to the slice number recorded in the file metadata, and record the slice number and the initial access heat value of each slice file into the file metadata. On this basis, the access heat value of each slice file in the first storage device is periodically calculated according to a preset time interval, and the access heat value of each slice file in the file metadata is updated by using the calculated access heat value to obtain an updated access heat value of each slice file, wherein the access heat value is used to determine whether each slice file is cold data.

[0033] It should be noted that the file processing method provided in the embodiments of the present application can generally be executed by the server 105. Correspondingly, the file processing apparatus provided in the embodiments of the present application can generally be arranged in the server 105. The file processing method provided in the embodiments of the present application can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the file processing apparatus provided in the embodiments of the present application can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the above-mentioned scenario is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

[0035] The file processing method provided in the embodiments of the present application will be described in detail below based on the scenario described above. Figure 1 Figures 2-3 The file processing method provided in the embodiments of the present application will be described in detail below based on the scenario described above.

[0036] Figure 2 A flowchart of the file processing method according to the embodiments of the present application is shown.

[0037] As shown in Figure 2 , the file processing method 200 includes operation S210 to operation S240.

[0038] In operation S210, the target file is sliced according to a preset slice size to obtain at least one slice file during writing the target file into the first storage device.

[0039] ​According to an embodiment of the present application, the first storage device represents a storage device for storing hot data, for example, the first storage device can be a high-performance storage medium. When writing a file into the storage device, the data is first written into the first storage device, and then it is judged whether the data is cold data based on the subsequent access heat value, so that the data is migrated out of the first storage device in the case of cold data.

[0040] According to an embodiment of the present application, during the process of writing the target file into the first storage device, the target file can be sliced according to a preset slice size to obtain at least one slice file.

[0041] In an embodiment, the preset slice size can be set as needed, and the preset slice size can represent the size of the slice file obtained after slicing. The target file can represent a file to be stored.

[0042] For example, the size of the target file is 100 MiB (Mebibyte, Megabit), and the preset slice size is set to 16 MiB. During the process of writing the target file into the first storage device, the target file can be divided according to 16 MiB, so that 7 slice files can be obtained. Among them, the size of the last slice file is only 4 MiB.

[0043] In an embodiment, if the second-to-last slice file of the target file is sliced, the size of the last part is less than the preset slice size, and the last part is also divided into a slice file.

[0044] In an embodiment, the target file is sliced from the starting position (0 MiB) of the target file.

[0045] In operation S220, at least one slice file is numbered according to the slice number recorded in the file metadata, and the slice number and the initial access heat value of each slice file are recorded in the file metadata.

[0046] According to an embodiment of the present application, after slicing the target file, at least one slice file needs to be numbered to distinguish different slice files.

[0047] In an embodiment, the slice number of each slice file written into the first storage device is recorded in the file metadata, so that the slice number recorded in the file metadata can be used to determine which number the slice file written into the first storage device has reached. On this basis, at least one slice file can be numbered.

[0048] Specifically, at least one slice file is numbered in order based on the slice number recorded in the file metadata.

[0049] For example, if the slice number 8 has been recorded in the file metadata, the at least one slice file is numbered, i.e., the at least one slice file is numbered from the number 9. If the target file is sliced into 3 slice files, the three slice files are numbered, i.e., the slice number 9, the slice number 10 and the slice number 11 are obtained respectively.

[0050] In an embodiment, the at least one slice file is numbered in sequence according to the generation order of the slice file.

[0051] For example, the target file is 48 MiB, and the preset slice size is set to 16 MiB. The target file is divided into slice files, and 0-16 MiB slice file, 17-32 MiB slice file and 33-48 MiB slice file are obtained in sequence. In the case that the slice number 8 has been recorded in the file metadata, the 0-16 MiB slice file can be numbered as the slice number 9, the 17-32 MiB slice file can be numbered as the slice number 10, and the 33-48 MiB slice file can be numbered as the slice number 11.

[0052] According to the embodiments of the present application, since the slice file will be accessed and determined whether it is cold data in the future, the slice number and the initial access hotness value of the at least one slice file can be recorded in the file metadata.

[0053] In an embodiment, the initial access hotness value can be set to 0.

[0054] In operation S230, the access hotness value of any slice file in the first storage device is periodically calculated according to a preset time interval.

[0055] According to the embodiments of the present application, the preset time interval is set according to the need. The access hotness value of any slice file in the first storage device is periodically calculated according to the preset time interval, i.e., the preset time interval is used as the period interval.

[0056] In an embodiment, the preset time interval can be set to 1 month, i.e., the access hotness value of any slice file in the first storage device is calculated once every 1 month.

[0057] The access hotness value can be used to represent the access condition of the slice file.

[0058] In operation S240, the access hotness value of any slice file in the first storage device is updated according to the calculated access hotness value, and the updated access hotness value of any slice file is obtained.

[0059] The access frequency value is used to determine whether the slice file is cold data.

[0060] According to the embodiment of the present application, the access frequency value of the slice file is recorded in the file metadata. Since the access frequency value of the slice file in the first storage device is calculated periodically, in order to accurately determine whether the current slice file is cold data based on the access frequency value, the access frequency value of the slice file in the file metadata needs to be updated after the access frequency value of the slice file is calculated each time, so that the determination of the slice file is based on the updated access frequency value of the slice file in the file metadata.

[0061] According to the embodiment of the present application, the target file written into the first storage device is dynamically divided into multiple slice files, and the calculation of the access frequency value is performed in units of slices, which realizes more fine-grained data management, i.e., determining whether the slice file is hot data or cold data in units of slices, so that the slice file can be used as the minimum migration unit in the subsequent process, and the waste of storage resources, system resources and bandwidth is avoided to a certain extent.

[0062] According to the embodiment of the present application, the file processing method further includes: in the case of receiving an access request for any slice file, updating the access frequency of the slice file in the file metadata and the access time corresponding to the access frequency.

[0063] According to the embodiment of the present application, the access frequency of the slice file in the first storage device and the access time corresponding to the access frequency are recorded in the file metadata. On this basis, the access frequency of the slice file in the file metadata and the access time corresponding to the access frequency need to be updated at the same time when the slice file is accessed.

[0064] In an embodiment, in the case of receiving an access request for any slice file, the slice file is accessed, i.e., the access frequency of the slice file is increased. The access time corresponding to the access frequency is the access time of the slice file at the access frequency.

[0065] For example, the access frequency of 5 is recorded for slice file A in the file metadata, so that in the case of receiving an access request for slice file A, the access frequency of slice file A in the file metadata needs to be updated to 5 times, and the access time corresponding to the access frequency of 5 is also recorded.

[0066] In an embodiment, the access time corresponding to the access frequency can be recorded in the form of a time stamp.

[0067] According to embodiments of this application, the file metadata also records the number of times a slice file is accessed and the corresponding access time. When any slice file is accessed, the number of times a slice file is accessed and the corresponding access time in the file metadata are updated simultaneously, which is convenient for subsequent calculation of the access popularity value of the slice file.

[0068] According to an embodiment of this application, periodically calculating the access popularity value of any slice file in the first storage device at preset time intervals includes: determining the number of accesses to any slice file within multiple preset time periods based on the number of accesses to any slice file in the file metadata and the access time corresponding to the number of accesses; and calculating the access popularity value for any slice file based on the number of accesses to any slice file within the multiple preset time periods and the respective weights of the multiple preset time periods in the file metadata.

[0069] According to an embodiment of this application, the access popularity value of any slice file in the first storage device is calculated at preset time intervals. Based on the number of accesses to any slice file and the access times corresponding to the accesses in the file metadata, the number of accesses to any slice file within multiple preset time periods can be determined.

[0070] In one embodiment, multiple preset time periods are set as needed. For example, multiple preset time periods may include within 2 hours, within 24 hours, or within 1 week.

[0071] In one embodiment, for any given slice file, the file metadata records the number of times that slice file was accessed and the corresponding access time, that is, it records the access time of each access to that slice file. Therefore, the number of times that slice file was accessed in multiple preset time periods can be calculated.

[0072] In one embodiment, the preset time period can be divided starting from the moment the access popularity value is calculated. Specifically, taking a preset time period of 2 hours as an example, the number of times the slice file is accessed within 2 hours from the moment the access popularity value is calculated is determined.

[0073] For example, for slice file A, the file metadata records the access time corresponding to access number 1 as June 21st, XX:YY:ZZ, access time corresponding to access number 2 as June 23rd, XX:YY:ZZ, and access time corresponding to access number 3 as June 29th, XX:YY:ZZ. For a preset time period of one week, if the access popularity value is calculated starting from June 29th, XX:YY:ZZ, then the access count for slice file within that preset time period is 2.

[0074] According to embodiments of this application, the file metadata also records the weights for multiple preset time periods. The weights for each preset time period are set as needed. However, the smaller the time span of a preset time period, the greater the importance of its corresponding access count as a reference. Therefore, the larger the time span of a preset time period, the smaller the corresponding weight needs to be set.

[0075] For example, the weight corresponding to a preset time period of 2 hours is greater than the weight corresponding to a preset time period of 1 week.

[0076] Based on this, the access popularity value for any slice file can be calculated according to the number of accesses to any slice file within multiple preset time periods and the weights of each preset time period in the file metadata. Specifically, the access counts for each preset time period are weighted and summed to obtain the access popularity value for the slice file.

[0077] For example, multiple preset time periods and their corresponding weights can be shown as follows: the number of visits within 2 hours represents recent visits, with a weight of 100; the number of visits within 24 hours represents visits within one day, with a weight of 80; the number of visits within 1 week represents visits within the most recent week, with a weight of 60; the number of visits within 1 month represents visits over a longer period of time, with a weight of 40; and the number of visits more than 1 month represents historical visits, with a weight of 20.

[0078] The formula for calculating the access popularity value of any slice file can be: Access popularity value = (number of visits in 2 hours × 100) + (number of visits in 24 hours × 80) + (number of visits in 1 week × 60) + (number of visits in 1 month × 40) + (number of visits in more than 1 month × 20).

[0079] According to embodiments of this application, the number of accesses to any slice file within multiple preset time periods can be determined by using the access counts and corresponding access times in the file metadata. Based on the different time spans of these preset time periods, different weights are assigned to each preset time period. Therefore, the access popularity value for any slice file can be calculated based on the access counts within these preset time periods and the respective weights of each preset time period in the file metadata. By considering the access counts within different preset time periods and the importance of time spans as a reference for access counts, assigning different weights to preset time periods with different time spans improves the effectiveness of the access popularity value to some extent, thus enhancing the accuracy of using the access popularity value to determine whether a slice file is hot or cold data.

[0080] According to an embodiment of this application, the above-mentioned file processing method further includes: determining any slice file as cold data when the access popularity value of any slice file in the file metadata is less than a preset access popularity value; migrating any slice file to a second storage device according to a preset migration address; and after the migration of any slice file is completed, deleting any slice file in the first storage device, updating the location information of any slice file in the file metadata to the preset migration address, and updating the migration status of any slice file to "migrated".

[0081] According to the embodiments of this application, after each calculation of the access popularity value for any slice file in the first storage device, it can be determined whether the slice file is cold data based on the access popularity value for any slice file in the file metadata, that is, the updated access popularity value for any slice file.

[0082] In one embodiment, the preset access popularity value is set as needed.

[0083] According to embodiments of this application, if the access popularity value of any slice file in the file metadata is greater than or equal to a preset access popularity value, then that slice file can be determined to be hot data. If the access popularity value of any slice file in the file metadata is less than the preset access popularity value, then that slice file can be determined to be cold data.

[0084] In another embodiment, any slice file can be defined as cold data when its access popularity value is less than or equal to a preset access popularity value; and any slice file can be defined as hot data when its access popularity value is greater than the preset access popularity value.

[0085] According to an embodiment of this application, if any slice file is cold data, it needs to be migrated from the first storage device to the second storage device. The second storage device can be a device used to store cold data; for example, it can be a low-cost storage medium. If the access popularity value of any slice file is greater than or equal to a preset access popularity value, the slice file can be determined to be hot data.

[0086] In one embodiment, the file metadata also stores a preset migration address, which points to an address on the second storage device. Specifically, any slice file identified as cold data can be migrated from the first storage device to the second storage device based on the preset migration address.

[0087] In one embodiment, during the process of writing the target file to the first storage device, after the target file is sliced, the file metadata can also record the location information of each slice file of the target file. At this time, the location information represents the address used to point to the first storage device.

[0088] According to the embodiments of this application, once the migration of any slice file is completed, that is, the slice file has been migrated to the second storage device, the slice file in the first storage device is cleared to free up space, and the location information of any slice file in the file metadata is updated to the preset migration address, and the migration status of any slice file is updated to "migrated".

[0089] In one embodiment, during the process of writing the target file to the first storage device, after the target file is sliced, the file metadata not only records the slice number and initial access popularity value of any slice file, but also records the migration status of the slice file. At this time, the migration status of the slice file is not migrated.

[0090] In one embodiment, the access popularity value of any slice file in the first storage device can be calculated periodically by a timer in the main process, that is, calculated periodically according to a preset time interval.

[0091] According to an embodiment of this application, when a slice file is determined to be cold data, the slice file can be marked as awaiting migration, and the combination of the file name and slice number corresponding to the slice file can be added to the migration queue. By adding the combination of the file name and slice number of all cold data existing in the first storage device to the migration queue, the cold data can be migrated.

[0092] According to embodiments of this application, a preset access popularity value is set to determine whether any slice file is cold or hot data based on the access popularity value of any slice file in the file metadata, thereby determining whether the slice file needs to be migrated. Thus, if a slice file is cold data, it is migrated from the first storage device to the second storage device, and the slice file is deleted from the first storage device. This avoids the waste of storage resources caused by keeping an entire large file on the first storage device for a small amount of hot data, reducing storage costs.

[0093] Figure 3 A schematic diagram of cold data migration according to an embodiment of this application is shown.

[0094] like Figure 3 As shown, the first storage device is an SSD, and the second storage device is an HDD. File system A is used to manage and use the first storage device, and file system B is used to manage and use the second storage device.

[0095] In one embodiment, file metadata can be managed on the first storage device side, i.e., file metadata is stored in file system A, which simplifies data management and scheduling.

[0096] The location information of any slice file in the first storage device recorded in the file metadata can be specifically a disk address, which can be mapped to a physical address, i.e., the physical address of the first storage device.

[0097] For example, such as Figure 3 As shown, the location information of slice file 1 recorded in the file metadata can point to the data of slice file 1 stored in the first storage device.

[0098] File systems A and B integrate NFS (Network File System) clients into their underlying read / write interfaces, and use the LibNFS (Library for Network File System) library to automatically mount and read / write cold data storage terminals via NFS sharing.

[0099] The second storage device is a cold data storage terminal.

[0100] In one embodiment, for slice files determined to be cold data, the preset migration address can be an NFS access path, i.e., an NFS shared path.

[0101] For example, such as Figure 3 As shown, for slice files 2 and 3, which are determined to be cold data, the slice files can be migrated to the second storage device via NFS, that is, the migrated slice files are saved in the specified NFS target path.

[0102] In one embodiment, for slice files identified as cold data, the file metadata records a field representing the NFS shared path. This field is specifically the absolute path plus the slice number. The absolute path indicates which file system the slice file was migrated from and to.

[0103] File system B records the filenames of slice files identified as cold data. These filenames are also absolute paths plus slice numbers. For example, the filename of slice file number 3, which is identified as cold data, is A.3. Here, A can represent the absolute path, indicating that the slice file was migrated from the file system, and 3 is the slice number.

[0104] For example, such as Figure 3 As shown, based on the file name A.3 recorded in file system B, it can point to the data of slice file 3 stored in the second storage device.

[0105] Therefore, records based on the NFS shared path field in the file metadata and the file name records of the sliced ​​files in file system B can maintain uniqueness and easy retrieval.

[0106] Based on this, cold data is stored in a specified NFS directory path, and the file metadata records the NFS access path. Therefore, the cold data storage terminal (second storage device) can be shared with the hot data storage device (first storage device), and it also supports specifying other storage locations across the network.

[0107] In one embodiment, cold data migration based on the NFS protocol is performed by a migration thread in the main program. The thread retrieves slice information from the queue to be migrated. The system uses the LibNFS library to establish a connection with the cold data storage terminal and mounts an NFS share. During migration, it reads the data of the slice from the hot data storage terminal and writes it to the path specified by the cold data terminal via the NFS protocol.

[0108] Therefore, the file processing method of this application supports higher deployment flexibility and data reliability. Specifically, by using the NFS protocol as the access interface for cold data storage, the cold data storage can be shared with the hot data storage, or other storage locations can be specified across the network. This supports geographically distributed hierarchical deployment of storage devices and improves data disaster recovery reliability.

[0109] According to an embodiment of this application, the above-described file processing method further includes: in response to a file access request, determining at least one slice number for the file access request from the file metadata based on the access information carried in the file access request; determining the migration status of at least one slice file to be accessed for the file access request from the file metadata based on the at least one slice number for the file access request; determining an access policy for each of the at least one slice file to be accessed based on the migration status of each of the at least one slice file to be accessed, and returning the read data to the requesting end by executing the access policy for each of the at least one slice file to be accessed.

[0110] According to embodiments of this application, upon receiving a file access request, the slice file corresponding to the file access request in the first or second storage device needs to be accessed. Based on this, at least one slice number corresponding to the file access request can be determined from the file metadata according to the access information carried in the file access request.

[0111] For example, based on the access information carried in the file access request, at least one slice number corresponding to the file access request can be determined from the file metadata, such as slice number 5, slice number 6, and slice number 7. That is, the slice files corresponding to slice number 5, slice number 6, and slice number 7 need to be accessed.

[0112] According to an embodiment of this application, the file metadata records the slice number of each slice file and its corresponding migration status. Therefore, the migration status of at least one slice file to be accessed for a file access request can be determined based on at least one determined slice number.

[0113] Since the unmigrated slice files are stored in the first storage device and the migrated slice files are stored in the second storage device, the access strategies for slice files in different migration states are different. Therefore, based on the migration state of each of the at least one slice file, an access strategy can be determined for each of the at least one slice files, and the read data can be returned to the requesting end by executing the access strategy for each of the at least one slice file.

[0114] In one embodiment, the requesting end represents the end that sends the file access request, and the requesting end can be, for example, a terminal, an application, etc.

[0115] According to embodiments of this application, upon receiving a file access request, it is necessary to determine the desired access slice file and its migration status, so as to determine different access strategies based on different migration statuses. This allows the execution of the access strategy for the slice file to read its data, and the read data can then be returned to the requesting end. The determination of different access strategies helps avoid invalid accesses.

[0116] According to an embodiment of this application, returning read data to the requesting end by executing an access policy for each of at least one slice to be accessed includes: when the migration status of each of the at least one slice to be accessed is not migrated, reading data of at least one slice to be accessed in a first storage device according to at least one slice number, and returning the read data to the requesting end.

[0117] According to an embodiment of this application, when the migration status of at least one slice file to be accessed is not migrated, the access strategy for each of the at least one slice file to be accessed is to directly read the data of the corresponding slice file to be accessed from the first storage device.

[0118] Since the migration status of the slice file to be accessed is not migrated, meaning the slice file to be accessed is stored in the first storage device, the data of the slice file to be accessed in the first storage device can be read directly according to the slice number of the slice file to be accessed.

[0119] In one embodiment, there is a one-to-one correspondence between the slice number and the slice file, that is, a slice file can be uniquely identified by the slice number.

[0120] According to an embodiment of this application, when the migration status of the slice file to be accessed is not migrated, the data of the slice file to be accessed can be directly read from the first storage device, and the read data can be returned to the requesting end.

[0121] According to an embodiment of this application, the above-mentioned file processing method further includes: when there is a target slice file in the migration status of at least one slice file to be accessed, determining the location information of the target slice file from the file metadata according to the slice number of the target slice file; based on the location information of the target slice file, migrating the target slice file from the second storage device to the first storage device, and clearing the target slice file in the second storage device; reading the data of at least one slice file to be accessed in the first storage device according to at least one slice number, and returning the read data to the requesting end.

[0122] According to an embodiment of this application, if at least one target slice file in the slice files to be accessed has a migration status of "migrated", and the target slice file is stored in a second storage device, then the access strategy for the target slice file is to migrate the target slice file from the second storage device to the first storage device, and read the data of the target slice file from the first storage device according to the slice number of the target slice file. The target slice file can be at least one slice file.

[0123] Specifically, the data of the target slice file is retrieved from the second storage device according to the NFS path using the LibNFS library.

[0124] In one embodiment, when a target slice file is migrated from a second storage device to a first storage device, the target slice file in the second storage device is deleted.

[0125] According to embodiments of this application, when a target slice file is migrated from a second storage device to a first storage device, data of the slice file to be accessed can be read from the first storage device based on the slice number of each of the at least one slice file to be accessed. This allows the data from the slice files of the first and second storage devices to be combined and returned to the requesting end.

[0126] In one embodiment, at least one slice file to be accessed includes slice file A and slice file B, where slice file A is in an unmigrated state and slice file B is in a migrated state. Therefore, slice file B can be migrated from the second storage device to the first storage device first, and data from slice files A and B can be read from the first storage device, allowing the read data to be returned to the requesting end.

[0127] In one embodiment, the process of real-time retrieval of cold data is transparent, that is, the process of retrieving the target slice file from the second storage device is transparent.

[0128] According to an embodiment of this application, when a target slice file in a migrated state exists within the slice file to be accessed, the target slice file is migrated from the second storage device to the first storage device based on its location information. Then, data from at least one slice file to be accessed is read from the first storage device based on its respective slice number, and the read data is returned to the requesting end. Since this is done on a slice file basis, only the target slice file needs to be retrieved from the second storage device, avoiding waste of system and bandwidth resources caused by large file sizes, and preventing potential impact on read / write speeds.

[0129] Since the migration and retrieval of cold data are carried out according to the minimum scope requirements (sliced ​​files), the waste of computing power can be effectively reduced and the storage space of high-performance storage media can be released in a timely manner.

[0130] According to an embodiment of this application, in response to a file access request, determining at least one slice number for the file access request from the file metadata based on the access information carried in the file access request includes: determining a range of slice numbers for the file to be accessed from the file metadata based on the file name of the file to be accessed; determining a starting slice number for the file access request from the range of slice numbers based on the access offset; determining a ending slice number for the file access request from the range of slice numbers based on the access length; and determining at least one slice number for the file access request based on the starting slice number and the ending slice number.

[0131] The access information may include the file name, access offset, and access length of the file to be accessed. The file metadata also records the file name of any file written to the first storage device.

[0132] According to an embodiment of this application, during the process of writing the target file to the first storage device, after the target file is sliced, the file metadata records the file name of the target file and the slice number of each of the multiple slice files obtained from slicing the target file. That is, the file name in the file metadata corresponds to the slice number of each of the target files of that file name.

[0133] Therefore, based on the file name of the file to be accessed, at least one slice number corresponding to the file name can be determined from the file metadata, that is, the range of slice numbers for the file to be accessed can be determined.

[0134] For example, for file A to be accessed, the file metadata records the slice numbers of file A, including slice number 5, slice number 6, slice number 7, slice number 8, and slice number 9. Therefore, the range of slice numbers for file A to be accessed is slice number 5 to slice number 9.

[0135] According to an embodiment of this application, the access offset represents the offset of the data to be accessed by the file access request relative to the starting position of the file to be accessed.

[0136] For example, if the access offset is 20 MiB, then the data that the file access request wants to access starts from the 20th MiB of the file to be accessed.

[0137] Therefore, based on the access offset, the starting slice number for the file access request can be determined from the slice number range. Specifically, since the preset slice size is known, the interval of the slice file corresponding to each slice number in the file to be accessed can be clearly defined. Thus, based on the access offset, it can be determined which slice file interval the access offset falls within, and consequently, the starting slice number for the file access request can be determined. The slice file corresponding to this starting slice number is the slice file corresponding to the interval in which the access offset falls.

[0138] For example, with a preset slice size of 10 MiB, the slice file corresponding to slice number 5 of the slice file A to be accessed has a slice size of 0~10 MiB, slice number 6 has a slice size of 11~20 MiB, slice number 7 has a slice size of 21~30 MiB, slice number 8 has a slice size of 31~40 MiB, and slice number 9 has a slice size of 41~50 MiB. With an access offset of 11 MiB, the starting slice number can be determined to be slice number 6.

[0139] According to embodiments of this application, the access length characterizes the amount of data that a file access request seeks to access.

[0140] For example, when the access offset is 20 MiB and the access length is 20 MiB, the data to be accessed by the file access request is 20 MiB of data starting from the 20 MiB position of the file to be accessed.

[0141] Therefore, based on the access length, the terminating slice number for the file access request can be determined from the slice number range. Specifically, since the preset slice size is known, the interval of the slice file corresponding to each slice number in the file to be accessed can be clearly identified. Based on the known access offset and the access length, it can be determined which interval of the file to be accessed is located in the file to be accessed, and thus the terminating slice number for the file access request can be determined.

[0142] For example, with an access offset of 11 MiB and an access length of 20 MiB, it can be determined that the data to be accessed by the file access request is located in the range of 11 MiB to 31 MiB of the file to be accessed. Since 31 MiB falls within the range corresponding to slice number 8, the terminating slice number for the file access request is slice number 8.

[0143] Based on this, at least one slice number can be determined for a file access request according to the starting slice number and the ending slice number. For example, if the starting slice number is slice number 6 and the ending slice number is slice number 8, at least one slice number for a file access request can be determined, namely slice number 6, slice number 7, and slice number 8.

[0144] According to embodiments of this application, since the data requested in a file access request may not be all the data in the file to be accessed, but may only be a portion of the data, at least one slice number can be determined for the file access request based on the file name, access offset, and access length carried in the file access request. This determines the slice file where the data requested in the file access request is located. Therefore, by using slice files as the unit, the slice file containing the data requested in the file access request can be accurately determined, avoiding unnecessary access to data in the file to be accessed.

[0145] According to an embodiment of this application, the above-mentioned file processing method further includes: in response to a trimming request for any file in the first storage device, determining the slice number of the cut file to be trimmed for any file from the file metadata based on the trimming information carried in the trimming request; based on the slice number of the cut file to be trimmed, clearing the cut file to be trimmed in the first storage device, and updating the number of slices and the slice number for any file in the file metadata.

[0146] According to embodiments of this application, the slice files of each file in the first storage device may change, such as by cropping a portion of the file data. Therefore, upon receiving a cropping request for any file, the slice number of the slice file to be cropped for any file can be determined from the file metadata based on the cropping information carried in the cropping request.

[0147] The cropping information can include the file name of the file to be cropped and the range of the data to be cropped. For example, if the file to be cropped is 100 MiB and the range of the data to be cropped is 50~100 MiB, then the data in the 50~100 MiB range of the file to be cropped should be cropped.

[0148] Specifically, based on the file name, the range of slice numbers corresponding to the file to be cropped can be determined from the file metadata. Based on the known preset slice size, the interval in which the slice file corresponding to each slice number is located can be determined, and then, based on the interval in which the data to be cropped is located, the slice number of the slice file to be cropped can be determined.

[0149] According to embodiments of this application, the file metadata also records the number of slices for any file. Based on the slice number for the file to be cut, the file to be cut in the first storage device can be cleared, and the number of slices and the slice number for any file in the file metadata can be updated.

[0150] In one embodiment, the file metadata also records the number of slices for any file.

[0151] According to an embodiment of this application, when the capacity of any file in the first storage device decreases, that is, when the cut pieces of any file have been cleared, it is necessary to update the number of pieces and the piece number corresponding to any file in the file metadata in a timely manner to ensure that the information recorded in the file metadata about any file matches the actual condition of any file.

[0152] In one embodiment, the file processing method of this application can be implemented based on the CephFS distributed file system. Through steps such as unified management of file metadata, automatic file slicing, calculation of access popularity values ​​of fine-grained sliced ​​files, cold data migration based on the NFS protocol, and real-time recall of sliced ​​files, fine-grained hierarchical management of hot and cold data can be achieved.

[0153] CephFS is the file system interface for the Ceph distributed storage system.

[0154] Unified management of file metadata can be reflected in: modifying the inode (index node) structure of file metadata, adding fields for preset slice size and number of slices, and adding a Slice structure to store slice file information, including slice number, slice file migration status, slice file location information, slice file access count and timestamp.

[0155] The inode is the core structure of the file system metadata, storing attributes such as file size, permissions, and timestamps, and pointing to the location of the data distribution.

[0156] Based on the above document processing method, this application also provides a document processing apparatus. The following will be combined with... Figure 4 The device is described in detail.

[0157] Figure 4 A structural block diagram of a document processing apparatus according to an embodiment of this application is shown.

[0158] like Figure 4 As shown, the file processing device 400 of this embodiment includes a slicing module 410, a numbering module 420, a calculation module 430, and an update module 440.

[0159] The slicing module 410 is used to slice the target file according to a preset slice size during the process of writing the target file to the first storage device, thereby obtaining at least one sliced ​​file. In one embodiment, the slicing module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0160] The numbering module 420 is used to number the at least one slice file according to the slice number recorded in the file metadata, and to record the slice number and initial access popularity value of each of the at least one slice file in the file metadata. In one embodiment, the numbering module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0161] The calculation module 430 is used to periodically calculate the access popularity value of any slice file in the first storage device at preset time intervals. In one embodiment, the calculation module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0162] The update module 440 is used to update the access popularity value for any of the above-mentioned slice files in the file metadata using the calculated access popularity value, thereby obtaining an updated access popularity value for any of the above-mentioned slice files. The access popularity value is used to determine whether any of the above-mentioned slice files is cold data. In one embodiment, the update module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0163] According to embodiments of this application, any plurality of modules among the slicing module 410, numbering module 420, calculation module 430, and update module 440 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the slicing module 410, numbering module 420, calculation module 430, and update module 440 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the slicing module 410, numbering module 420, calculation module 430, and update module 440 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0164] Figure 5 A block diagram of an electronic device suitable for implementing a file processing method according to an embodiment of this application is shown.

[0165] like Figure 5 As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0166] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0167] According to embodiments of this application, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0168] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0169] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 502 and / or RAM 503 and / or one or more memories other than ROM 502 and RAM 503 described above.

[0170] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the file processing method provided in the embodiments of this application.

[0171] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0172] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0173] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0174] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0176] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0177] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A file processing method, characterized in that, The file processing method includes: During the process of writing the target file to the first storage device, the target file is sliced ​​according to a preset slice size to obtain at least one slice file; Based on the slice number recorded in the file metadata, the at least one slice file is numbered, and the slice number and initial access popularity value of each of the at least one slice file are recorded in the file metadata; The access popularity value of any slice file in the first storage device is periodically calculated according to a preset time interval. Using the calculated access popularity value, update the access popularity value for any slice file in the file metadata to obtain the updated access popularity value for any slice file, wherein the access popularity value is used to determine whether any slice file is cold data; The file processing method further includes: In response to a file access request, at least one slice number for the file access request is determined from the file metadata based on the access information carried in the file access request. Based on at least one slice number for the file access request, determine the migration status of at least one slice file to be accessed for the file access request from the file metadata; If a target slice file with a migration status of "migrated" exists in at least one of the slice files to be accessed, the location information of the target slice file is determined from the file metadata based on the slice number of the target slice file. Based on the location information of the target slice file, the target slice file is migrated from the second storage device to the first storage device, and the target slice file in the second storage device is deleted; Data of at least one slice file to be accessed in the first storage device is read according to the at least one slice number, and the read data is returned to the requesting end.

2. The file processing method according to claim 1, characterized in that, The file processing method further includes: Upon receiving an access request for any of the slice files, update the file metadata with the number of accesses for any of the slice files and the access time corresponding to the number of accesses.

3. The file processing method according to claim 2, characterized in that, The step of periodically calculating the access popularity value of any slice file in the first storage device at preset time intervals includes: According to the preset time interval, based on the number of accesses to any slice file in the file metadata and the access time corresponding to the number of accesses, determine the number of accesses to any slice file within multiple preset time periods; The access popularity value for any slice file is calculated based on the number of accesses to any slice file within the multiple preset time periods and the respective weights of the multiple preset time periods in the file metadata.

4. The file processing method according to claim 1, characterized in that, The file processing method further includes: If the access popularity value of any slice file in the file metadata is less than a preset access popularity value, then any slice file is identified as cold data. According to the preset migration address, any of the slice files will be migrated to the second storage device; Upon completion of the migration of any of the slice files, the slice file is cleared from the first storage device, and the location information of the slice file in the file metadata is updated to the preset migration address, and the migration status of the slice file is updated to "migrated".

5. The document processing method according to claim 4, characterized in that, The file processing method further includes: Based on the migration status of each of the at least one slice file to be accessed, an access strategy is determined for each of the at least one slice file to be accessed, and the read data is returned to the requesting end by executing the access strategy for each of the at least one slice file to be accessed.

6. The document processing method according to claim 5, characterized in that, The step of returning the read data to the requesting end by executing the access strategy for each of the at least one slice to be accessed includes: If the migration status of each of the at least one accessed slice file is not migrated, the data of the at least one accessed slice file in the first storage device is read according to the at least one slice number, and the read data is returned to the requesting end.

7. The document processing method according to claim 5, characterized in that, The access information includes the file name, access offset, and access length of the file to be accessed, and the file metadata also records the file name of any file written to the first storage device. In response to a file access request, determining at least one slice number for the file access request from the file metadata based on the access information carried in the file access request includes: Based on the file name of the file to be accessed, determine the range of slice numbers for the file to be accessed from the file metadata; Based on the access offset, determine the starting slice number for the file access request from the slice number range; Based on the access length, determine the termination slice number for the file access request from the slice number range; Based on the starting slice number and the ending slice number, at least one slice number is determined for the file access request.

8. The document processing method according to any one of claims 1 to 7, characterized in that, The file processing method further includes: In response to a trimming request for any file in the first storage device, the slice number of the file to be trimmed for any file is determined from the file metadata based on the trimming information carried in the trimming request. Based on the slice number of the cut-out slice file, the cut-out slice file in the first storage device is cleared, and the slice count and slice number for any file in the file metadata are updated.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images