File processing methods, apparatus, devices, and storage media

Abstract

The present invention provides a file processing method, apparatus, device, and storage medium, which determine the segment order of each file segment by arrangement parameters of the file segments, and determine the target position of each file segment in the target reserved area based on the segment order of each file segment, thereby ensuring that each file segment written to the target reserved area is contiguous and further mitigating the problem of internal file fragmentation.

Description

Technical Field

[0001] This application claims the priority of a Chinese patent application with an application date of April 14, 2023, an application number of 202310410411.7, and an invention title of "File Processing Method, Apparatus, Device, and Storage Medium", and the entire content of the application is incorporated herein by reference.

[0002] This application relates to the technical field of computer information storage, and particularly to file processing methods, apparatuses, devices, and storage media.

Background Art

[0003] From the log-structured file system, various file systems, collectively called log-structured (LFS) file systems, such as F2FS and NILFS, have been derived. The main characteristics of this type of file system are the out-of-place update mechanism and the checkpoint mechanism. Whether the underlying storage device is a mechanical hard disk or a flash memory device, the out-of-place update mechanism can significantly improve the random write performance. Therefore, this type of file system is widely applied to smart mobile devices.

[0004] The side effect of the out-of-place update mechanism is that compared with other in-place update file systems, file fragmentation caused by file updates in the LFS type of file system significantly reduces the sequential read and sequential write performance of the structured file system.

Summary of the Invention

Problems to be Solved by the Invention

[0005] Conventional methods primarily mitigate the fragmentation problem of structured file systems through the garbage collection (GC) mechanism. However, this mechanism can only mitigate external file fragmentation problems and cannot mitigate internal file fragmentation problems. [Means for solving the problem]

[0006] Embodiments of the present invention provide a file processing method, apparatus, device, and storage medium that mitigate the internal file fragmentation problem of a structured file system.

[0007] In one embodiment, the present invention provides a file processing method, and the method is Obtain the configuration parameters of each file segment in the file to be processed, and determine the target reserved area of ​​the file to be processed. The segment order of each file segment is determined based on the placement parameters of each file segment, Based on the segment order of each file segment, the target position of each file segment in the target reserved area is determined, This includes writing each of the aforementioned file segments to the corresponding target location in the aforementioned target reserved area.

[0008] In another embodiment, an embodiment of the present invention provides a file processing device, the device is An acquisition module for obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed, A fragment sorting module for determining the segment order of each file segment based on the placement parameters of each file segment, A fragmentation-sensitive file address mapping module for determining the target position in the target reserved area of ​​each file segment based on the segment order of each file segment, The system includes an internal anti-fragmentation recovery module for writing each of the file segments to a target location in the corresponding target reserved area.

[0009] In another embodiment, an embodiment of the present invention provides a file processing device comprising memory and a processor, wherein an application program is stored in the memory, and the processor is used to execute the application program in the memory in order to perform the operations in the above-described method.

[0010] In another embodiment, an embodiment of the present invention provides a storage medium having a plurality of instructions stored therein, which are adapted to be loaded by a processor to perform the steps in the above method. [Effects of the Invention]

[0011] The file processing method, apparatus, device, and storage medium provided by embodiments of the present invention relate to the technical field of computer information storage and involve obtaining the arrangement parameters of each file segment in a file to be processed, determining a target reserved area for the file to be processed, determining the segment order of each file segment based on the arrangement parameters of each file segment, determining the target position of each file segment in the target reserved area based on the segment order of each file segment, and writing each file segment to the corresponding target position in the target reserved area. In embodiments of the present invention, by determining the segment order of each file segment based on the arrangement parameters of each file segment and determining the target position of each file segment in the target reserved area based on the segment order of each file segment, it is ensured that each file segment written to the target reserved area is contiguous, thereby mitigating the problem of internal file fragmentation.

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used to describe the embodiments are briefly introduced below. Obviously, the drawings described below represent only a few embodiments of the present invention. Those skilled in the art can obtain other drawings based on these without any creative effort. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of the out-of-place update mechanism of prior art. [Figure 2] This is a schematic diagram of a conventional garbage collection mechanism from prior art. [Figure 3] This is a schematic diagram of one embodiment of the file processing method according to the present invention. [Figure 4] This is a schematic diagram of another embodiment of the file processing method according to the present invention. [Figure 5] This is a schematic diagram of an embodiment of the method for determining internal fragmentation data according to the present invention. [Figure 6] This is a schematic diagram of an embodiment of the method for determining the target reserved area of ​​a file to be processed according to the present invention. [Figure 7] This is a schematic diagram of an embodiment of the method for determining the target position in the target reservation area of ​​a file segment according to the present invention. [Figure 8] This is a schematic diagram of the file internal fragmentation process according to the present invention. [Figure 9] This is a schematic diagram of another embodiment of the file processing method according to the present invention. [Figure 10] This is a schematic diagram of another embodiment of the file processing method according to the present invention. [Figure 11] This is a schematic diagram of another embodiment of the file processing method according to the present invention. [Figure 12] This is a schematic diagram of an embodiment of the method for updating a list to be organized according to the present invention. [Figure 13] This is a schematic diagram illustrating how a file according to the present invention is written to a structured file system. [Figure 14]It is a schematic diagram of an embodiment of a file processing apparatus according to the present invention. [Figure 15] It is a schematic diagram of an embodiment of a file processing device according to the present invention.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application and do not represent all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are included in the protection scope of the present disclosure.

[0015] As described in the background art, in an out-of-place update mechanism, when deleting, overwriting, or updating parameters of a file segment in a file stored in a structured file system, file internal fragmentation and file external fragmentation occur following the discontinuity of file segments in the file and the discontinuity of free storage areas in the structured file system. Here, file internal fragmentation is expressed as the discontinuity of file segments in the file, and file external fragmentation is expressed as the discontinuity of free storage areas in the structured file system, that is, the fragmentation of the remaining area of the structured file system.

[0016] As an example, as shown in Figure 1, Figure 1 is a schematic diagram of an out-of-place update mechanism in the prior art, and the file structure system shown in Figure 1 contains 16 storage areas. As shown in Figure 1(a), in the initial state, file A contains four file segments A1, A2, A3 and A4, and file A occupies four storage areas contiguously in the file structure system. File B contains four file segments B1, B2, B3 and B4, and file B occupies four storage areas contiguously in the file structure system. The remaining area of ​​the structured file system is contiguous. When out-of-place updates are performed sequentially on file segments A2, B3, A2, and B2, the updated file segment A2 is written to free storage area 08, the updated file segment B3 is written to free storage area 09, the updated file segment A2 is written to free storage area 10, and the updated file segment B2 is written to free storage area 11, while the file segments stored in storage areas 01, 03, 05, and 06 are erased. As shown in Figure 1(b), the file segments of file A and file B are no longer contiguous, causing internal file fragmentation. Although the number of storage areas in the remaining area of ​​the structured file system does not change, the storage areas in the remaining area of ​​the structured file system are no longer contiguous, causing external file fragmentation. When it is necessary to read file A and file B sequentially, the number of I / Os and overhead increases because the file segments of file A and file B are not contiguous. At the same time, the storage addresses of the file segments of file A and file B become out of order, which degrades the performance of sequentially addressable storage devices. If the C file to be written contains eight file segments, then file segment C1 of the C file is written to the remaining storage area 12, file segment C2 is written to the remaining storage area 13, file segment C3 is written to the remaining storage area 14, and file segment C4 is written to the remaining storage area 15.When writing file segment C5, since there is no free storage space after storage area 15, it is necessary to traverse the storage space of the structured file system, determine the free storage area 01 from the storage space of the entire structured file system, write file segment C5 to storage area 01, and then sequentially write file segments C6, C7, and C8 according to the writing method of file segment C5. As a result, the number of I / O operations during file writing increases, and as shown in Figure 1(c), the file segments of the written C file become discontinuous, causing fragmentation in the internal file structure.

[0017] Prior art designs garbage collection mechanisms to mitigate the fragmentation problem of structured file systems. However, existing garbage collection mechanisms can only mitigate fragmentation of the remaining space in the structured file system, and cannot mitigate internal file fragmentation. Exemplarily, as shown in Figure 2, Figure 2 is a schematic diagram of a conventional garbage collection mechanism. As shown in Figure 2(a), in a conventional garbage collection mechanism, the structured system file is divided into four storage segments sec1, sec2, sec3, and sec4, each storage segment containing four storage areas, and files A and B have both external and internal file fragmentation. As shown in Figures 2(b) and (c), in a conventional garbage collection mechanism, each storage segment is traversed, and the data from the storage area used in each storage segment is written to a contiguous storage segment in the free storage area, thereby making the remaining storage area in the structured system file contiguous. As shown in Figure 2(d), in a structured file system organized by a garbage collection mechanism, the file segments of file A and file B are still not contiguous, indicating that conventional garbage collection mechanisms cannot mitigate internal file fragmentation. When files A and B need to be read sequentially, internal file fragmentation of files A and B increases the number of I / Os and overhead, degrading the performance of sequentially addressable storage devices.

[0018] Based on this, in order to mitigate the problem of internal file fragmentation, embodiments of the present invention provide a file processing method, apparatus, device, and storage medium that ensure that each file segment written to the target reserved area is contiguous by determining the segment order of each file segment by arrangement parameters of the file segments and determining the target position of each file segment in the target reserved area based on the segment order of each file segment, thereby further mitigating the problem of internal file fragmentation.

[0019] The following describes in detail the implementation details of the technical proposal of the embodiment of the present invention.

[0020] As shown in Figure 3, Figure 3 is a schematic diagram of one embodiment of the file processing method according to the present invention. The method may be performed by electronic devices having data storage and data processing capabilities, such as servers, smartphones, tablet computers, and personal computers, and is not specifically limited thereto.

[0021] As shown in Figure 3, the method includes at least steps 301 to 304, the details of which are described below.

[0022] Step 301: Obtain the configuration parameters of each file segment in the file to be processed and determine the target reserved area of ​​the file to be processed.

[0023] The file to be processed in this invention may be a file to be written to a structured file system. For example, when a file write request is received, a file write operation is triggered to obtain the file to be processed, and the file segments of the file to be processed are traversed to obtain the configuration parameters of each file segment. The file to be processed includes, but is not limited to, the initial write file, the file segment or file to be updated, or a file that has already been written in the structured file system.

[0024] Selectively, in some embodiments, if the file write operation is the first write to a file, the file to be written is set as the file to be processed, and each file segment of the file to be processed is traversed to determine the configuration parameters of each file segment.

[0025] In some other embodiments, selectively, if the file write operation is an update of a file segment, a written file segment in the structured file system that matches the file identifier of the file segment to be updated is determined, the set consisting of the file segment to be updated and the file segments in the written file that match the file identifier is set as the file to be processed, and the configuration parameters of each file segment are determined by traversing each file segment in the file to be processed. Here, a file segment update represents performing a data update on at least one file segment of a written file in the structured file system. A data update includes, but is not limited to, overwriting a file segment or performing a data update on a file segment.

[0026] In some other embodiments, selectively, if a file write operation is a file update, the file to be updated is retrieved, the file to be updated is set as the file to be processed, and each file segment in the file to be processed is traversed to determine the configuration parameters of each file segment. Here, a file update represents overwriting at least one written file in the structured file system.

[0027] In some other embodiments of the present invention, selectively, the file to be processed may be a written file in the structured file system. For example, upon receiving a defragmentation request, a file defragmentation operation is triggered, a target written file is selected from the structured file system, the selected target written file is set as the file to be processed, and each file segment of the file to be processed is traversed to determine the configuration parameters of each file segment. Here, the target written file may be any written file in the structured file system, or it may be selected from written files in the structured file system based on predetermined conditions. Here, predetermined conditions include, but are not limited to, file size, file type, and number of file updates.

[0028] The configuration parameters in this invention represent the order of each file segment in the file to be processed, and include, but are not limited to, the segment code, creation time, and modification time of each file segment in the file to be processed.

[0029] In the present invention, the target reserved area is a contiguous storage area allocated to a file to be processed in a structured file system, for storing only the file to be processed. It includes, but is not limited to, persistent storage and temporary storage. Here, persistent storage indicates that the target reserved area is used only to store the file to be processed and that any free reserved storage in the target reserved area is not reclaimed, while temporary storage indicates that the target reserved area is used only to store the file to be processed and that any free reserved storage in the target reserved area can be reclaimed.

[0030] Here, "reclaiming the free reserved storage area in the target reserved area" means that, when the remaining space of the structured file system is insufficient, the free reserved storage area in the target reserved area is reclaimed as free storage space, thereby increasing the remaining space of the structured file system. Here, the remaining space of the structured file system refers to the storage area consisting of the free storage area in the structured file system, and free storage area refers to the normal storage area in the structured file system where no data has been written and which does not belong to a reserved area.

[0031] In some embodiments of the present invention, a file identifier for a file to be processed can be obtained, and a reserved area in the structured file system that matches the file identifier can be determined as the target reserved area for the file to be processed. Here, the file identifier may be one or more of the alphabet, numbers, and special characters, and the file identifier may be used to represent the file type of the file to be processed, and the file type includes, but is not limited to, video files, text files, audio files, and image files. Exemplarily, a contiguous free storage area in a file structured system is divided into a plurality of reserved areas, a predetermined file type is determined for each reserved area, the file identifier for the file to be processed is determined, a mapping relationship between the predetermined file identifier and the file type is searched to determine a predetermined file type corresponding to the file type of the file to be processed, and based on the predetermined file type corresponding to each reserved area, a reserved area in which the predetermined file type matches the file type of the file to be processed is determined as the target reserved area for the file to be processed.

[0032] In some other embodiments of the present invention, a data isolation mechanism may be used to determine contiguous free storage areas in the storage area of ​​a structured file system as the target reserved area for the file to be processed. For example, a contiguous free storage area in a structured file system may be divided into at least two storage areas, a first storage area and a second storage area, with the storage area in the first storage area set as the normal storage area and the second storage area set as the target reserved area for the file to be processed. Here, the data isolation mechanism means that when writing, updating, or erasing data in the target reserved storage area in the target reserved area, it does not affect data in the normal storage area in the first storage area.

[0033] Step 302: Determine the segment order of each file segment based on the placement parameters of each file segment.

[0034] The segment order of a file segment is the sequence number of the file segment in the file segment sequence, which is obtained by sorting each file segment based on the placement parameters of each file segment.

[0035] In some embodiments, each file segment may be sorted in descending order of configuration parameters, or in ascending order of configuration parameters.

[0036] Step 303: Based on the segment order of each file segment, determine the target position of each file segment within the target reserved area.

[0037] The target location may be the starting address of the free reserved storage area in the target reserved area, or it may be the address range of the free reserved storage area in the target reserved area. Here, the address refers to the physical address where the data is stored in the structured file system.

[0038] In some embodiments, selectively, based on the segment order of a file segment, the target position associated with each segment order can be obtained by retrieving pre-stored relationship data between segment orders and positions, and the target position associated with each segment order can be determined as the target position in the target reserved area of ​​the file segment corresponding to that segment order. Here, the relationship data between segment orders and positions includes a plurality of segment orders and the position corresponding to each segment order.

[0039] In some other embodiments, selectively, a contiguous free reserved storage area in the target reserved area is determined, the segment order of each file segment is traversed, and the address range or starting address of the first free reserved storage area in the contiguous free reserved storage area is set as the target position in the target reserved area of ​​the file segment corresponding to the first segment order, starting from the first free reserved storage area until the traversal of the segment order of each file segment is complete, and the address range or starting address of the next adjacent free reserved storage area of ​​the first free reserved storage area is set as the target position in the target reserved area of ​​the file segment corresponding to the second segment order.

[0040] Here, the first free reserved storage area may be the free reserved storage area with the lowest address range or starting address among the contiguous free reserved storage areas.

[0041] Step 304: Write each file segment to its corresponding target location in the target reserved area.

[0042] In this invention, after determining the target position in the target reserved area of ​​each file segment, each file segment is written to the corresponding target position in the target reserved area.

[0043] Optionally, in some embodiments, when the file to be processed is the file to be written, each file segment of the file to be processed is sequentially written to the target position corresponding to the file segment in the target reservation area according to the segment order of each file segment. For example, if the file to be processed includes four file segments A1, A2, A3, and A4, and the segment order corresponding to each of A1, A2, A3, and A4 is A1 < A2 < A3 < A4, then A1, A2, A3, and A4 are sequentially written to the target positions in the target reservation areas corresponding to A1, A2, A3, and A4 respectively.

[0044] Optionally, in some embodiments, when the file to be processed is a written file, according to the segment order of each file segment, the file segments corresponding to each segment order are sequentially copied, and the copied file segments corresponding to the segment order are written to the target position in the target reservation area of the file segment corresponding to the segment order.

[0045] In the file processing method according to the embodiment of the present invention, the segment order of each file segment is determined by the arrangement parameter of the file segment, and based on the segment order of each file segment, the target position in the target reservation area of each file segment is determined, so as to ensure that each file segment written in the target reservation area is continuous, and further reduce the problem of internal fragmentation of the file.

[0046] Based on the embodiment of FIG. 3, in order to improve the system performance, the fragmentation degree of the file to be processed can be processed. When the requirement of the fragmentation degree is met, the process of the fragmentation process is executed. FIG. 4 is a schematic diagram of another embodiment of the file processing method according to the present application. Another embodiment of the file processing method according to the present application shown in FIG. 4 includes at least steps 401 to 406.

[0047] Step 401, determine the internal fragmentation data of the file to be processed.

[0048] In this embodiment, the files to be processed written to the structured file system may be files that have been written to the structured file system within a certain period in the past, where this period may be the past day, the past three days, or the past week. In this embodiment, the files to be processed written to the structured file system may be files that have been written to the structured file system within a certain period in the past and that satisfy predetermined rules, where the predetermined rules may be one or more of file tags, file size, and file type. For example, if the predetermined rule is file tags, the files to be processed may be predetermined files that have file tags that have not undergone internal fragmentation processing, or predetermined files that have file tags that are fragmented files.

[0049] In this embodiment, the internal fragmentation data represents the degree of internal file fragmentation of the file to be processed, and includes, but is not limited to, the degree of out-of-order file segments and the degree of holes in file segments in the file to be processed.

[0050] In some embodiments of the present invention, the internal fragmentation data of a file to be processed can be determined based on the storage address of the file segment of the file to be processed. Specifically, as shown in Figure 5, the present invention provides a method for determining internal fragmentation data, which specifically includes steps 4011 to 4014.

[0051] Step 4011: Obtain the storage address of the file segment of the file to be processed.

[0052] Here, the storage address may be the starting address of the physical address in the file segment's structure file system, or it may be an address range of the physical address in the file segment's structure file system. In some embodiments, the storage address of the file segment of the file to be processed can be read directly from the structure file system, and in some other embodiments, the logical address of the file segment of the file to be processed can be read from the structure file system, logical storage address mapping information for the file to be processed can be obtained, and the storage address of the file segment of the file to be processed can be obtained based on the logical storage address mapping information for the file to be processed. Here, the logical storage address mapping information includes the mapping relationship between logical addresses and storage addresses.

[0053] Selectively, in some embodiments, considering that the degree of internal fragmentation of the file to be processed is low when it is first written to the structured file system, when a file to be processed is written to the structured file system and the file write operation is the first write to the file, the file to be processed may be written directly to the remaining space in the structured file system, or a target reserved area for the file to be processed may be determined, and based on the configuration parameters of the file segments in the file to be written, the file segments in the file to be processed may be sequentially written to the free reserved storage area in the target reserved area.

[0054] Selectively, in some embodiments, the data of the file to be processed in a non-first write structure file system takes into account that the file write operation is an update of a file segment, and internal fragmentation occurs in the file segment of the file to be processed due to factors such as the update of the file segment of the file to be processed and discontinuities in the remaining space in the structure file system. Therefore, the storage address in the structure file system of the file segment of the file to be processed is obtained, and steps 4012 to 4014 are performed to determine the internally fragmented data of the file to be processed.

[0055] Step 4012: Based on the storage address of each file segment, determine the discrete data of the file to be processed.

[0056] Here, discrete data is used to quantize the degree of disorder in the file segments of the file to be processed.

[0057] In some embodiments, discrete data for a file to be processed can be obtained based on the logical address range of the file to be processed and the storage address of each file segment. Here, the discrete data may be the number of file segments whose storage addresses are out of order. For example, if the logical address range of file A is from LAB10 to LAB13, and the corresponding storage addresses are PPN10, PPN12, PPN11, and PPN13, and the logical address range of file B is from LAB14 to LAB17, and the corresponding storage addresses are PPN14 to PPN17, and the storage addresses of the file segments in file A whose logical addresses are LAB11 and LAB12 are out of order, then it is determined that the discrete data for file A is 2 and the discrete data for file B is 0.

[0058] In some other embodiments, based on the configuration parameters of each file segment in the file to be processed and the storage address of each file segment, the number of file segments in the file to be processed whose storage addresses are out of order can be statistically determined, and discrete data of the file to be processed can be obtained. Specifically, the discrete data determination method includes at least steps 40121 to 40123.

[0059] Step 40121: Determine the segment order of each file segment based on the configuration parameters of each file segment in the file to be processed.

[0060] Step 40122: Determine the storage order of each file segment based on the storage address of each file segment in the file to be processed.

[0061] Step 40123: Based on the difference information between the storage order of each file segment and the segment order, it is decided to retrieve the discrete data of the file to be processed.

[0062] Selectively, in some embodiments, the difference information may be the order difference or similarity between the stored order of each file segment and the segment order. Exemplarily, taking the difference information as the order difference between the stored order of each file segment and the segment order, the order difference between the stored order of the file segment and the segment order can be determined for each file segment. If the order difference is non-zero, it is determined that the stored order of the file segment does not match the segment order of the file segment; if the order difference is zero, it is determined that the stored order of the file segment matches the segment order of the file segment. The number of file segments in the file to be processed whose stored order does not match the segment order of the file segment is statistically calculated to obtain discrete data for the file to be processed. For example, if the configuration parameter is a segment code, and the segment codes corresponding to file segments A1, A2, A3, and A4 of file A are 001, 002, 003, and 004 respectively, and the segment orders corresponding to file segments A1, A2, A3, and A4 of file A are 1, 2, 3, and 4 respectively, and the storage orders corresponding to file segments A1, A2, A3, and A4 of file A are 4, 3, 2, and 1 respectively, that is, if the storage orders of each file segment in file A do not match the segment order of the file segment, then it is determined that the discrete data for the file to be processed is 4. If the storage orders corresponding to file segments A1, A2, A3, and A4 of file A are 1, 2, 3, and 4 respectively, that is, if the storage orders of each file segment in file A match the segment order of the file segment, then it is determined that the discrete data for the file to be processed is 0.

[0063] Step 4013: Based on the state difference information between each file segment and its adjacent file segments, determine the file's whole data to be processed.

[0064] Here, state difference information represents the difference between the state information of each file segment and the state information of adjacent file segments of that file segment. Hole data is used to quantize the hole degree of the file to be processed, where the hole degree is used to determine whether all file segments in the file to be processed are stored in a series of contiguous memory areas. If the hole degree is not 0, it is determined that all file segments of the file to be processed are not stored in a series of contiguous memory areas, and if the hole degree is 0, it is determined that all file segments of the file to be processed are stored in a series of contiguous memory areas. The state information includes the file identifier of the file in which the file segment is located and the storage state of the file segment, and the state status includes stored and free.

[0065] In step 4013, the state information between each file segment and its adjacent file segments is compared to determine the state difference information between each file segment and its adjacent file segments, and the whole data of the file to be processed is determined based on the state difference information. Specifically, the method for determining the whole data of the file to be processed includes at least steps 40131 to 40133.

[0066] Step 40131: For each file segment, compare the status information of the file segment with the status information of its adjacent file segments.

[0067] Step 40132: Determine the number of adjacent file segments that do not match the status information of the file segment as the local hole data for the file segment.

[0068] Step 40133: Based on the local hole data of each file segment, obtain the hole data for the file to be processed.

[0069] Here, the adjacent file segment may be the file segment immediately preceding the file segment that is adjacent to it, or it may be the file segment immediately following the file segment that is adjacent to it.

[0070] As an example, let's explain using the case where an adjacent file segment is the file segment immediately preceding the file segment in question. Starting with the first file segment of the file to be processed, the state information of that file segment is compared with the state information of its adjacent file segment. If the state information of the file segment matches the state information of its adjacent file segment, the local hole data of that file segment is determined to be 0, and the local hole data of the next file segment in question is determined. If the status information of the file segment does not match the status information of the adjacent file segment of the file segment, the next adjacent file segment of the adjacent file is set as the new adjacent file segment, the status information of the file segment is compared with the status information of the new adjacent file, and if the status information of the file segment matches the status information of the new adjacent file, the local hole data of the file segment is determined to be 1, and the local hole data of the next file segment of the file segment is determined, and if the status information of the file segment does not match the status information of the new adjacent file segment, the status information of the file segment is compared with the status information of the next file segment of the new adjacent file segment, and so on, until the status information of the new adjacent file matches the status information of the file segment, the number of adjacent file segments whose status information does not match the status information of the file segment is statistically calculated, the local hole data is obtained, and the local hole data of the next file segment of the file segment is determined.

[0071] In some embodiments, selectively, local hole data for each file segment can be aggregated to obtain hole data for the file to be processed.

[0072] In some other embodiments, the local hall data of each file segment may be statistically analyzed to obtain statistical features of the local hall data, and these statistical features may be set as the hall data of the file to be processed. Here, the statistical features include, but are not limited to, the mean, maximum, and polar difference values ​​of the local hall data.

[0073] Step 4014: Based on the discrete and whole data of the file to be processed, it is decided to obtain the internal fragmentation data of the file to be processed.

[0074] In some embodiments, the discrete and whole data of the file to be processed can be combined to obtain the internal fragmentation data of the file to be processed.

[0075] In some other embodiments, selectively, the arithmetic mean or weighted mean of the discrete data and whole data of the file to be processed can be determined as the internal fragmentation data of the file to be processed.

[0076] In some other embodiments, selectively, the sum of weights corresponding to the discrete data and whole data of the file to be processed may be obtained, and the internal fragmentation data of the file to be processed may be determined based on the discrete data and whole data of the file to be processed, and the weighted sums corresponding to the discrete data and whole data of the file to be processed.

[0077] Step 402: If the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, the system retrieves the configuration parameters of each file segment in the file to be processed and determines the target reserved area of ​​the file to be processed.

[0078] In this embodiment, after obtaining the internal fragmentation data of the file to be processed, the internal fragmentation data of the file to be processed is compared with a predetermined internal fragmentation threshold. If the internal fragmentation data of the file to be processed is greater than or equal to the predetermined internal fragmentation threshold, it is indicated that the degree of internal fragmentation of the file to be processed is high. The configuration parameters of each file segment of the file to be processed are then obtained, and the target reserved area of ​​the file to be processed is determined.

[0079] In this embodiment, the target reserved area of ​​a file to be processed can be determined based on the file size, cumulative number of writes, and creation time of the file to be processed. Specifically, as shown in Figure 6, the present invention further provides a method for determining the target reserved area of ​​a file to be processed, which includes steps 4021 to 4023.

[0080] Step 4021: Based on the time interval between the creation time of the file to be processed and the current time, and the cumulative number of writes to the file to be processed, determine the editing frequency of the file to be processed within the time interval.

[0081] In some embodiments, the frequency of editing files to be processed within an interval can be obtained by the cumulative number of writes / interval time.

[0082] Step 4022: Based on the editing frequency and the file size of the files to be processed, determine the number of reserved storage areas for the files to be processed.

[0083] In some embodiments, selectively, the required storage space size for the files to be processed may be obtained by using the product of the editing frequency and the file size of the files to be processed, the storage space size of each predetermined reserved storage space may be obtained, and the number of reserved storage spaces for the files to be processed may be obtained by dividing the storage space size by the storage space size of each reserved storage space.

[0084] In some other embodiments, selectively, the storage size of each predetermined reserved storage area is obtained, the initial number of reserved storage areas for files to be processed is obtained by the file size of the file to be processed / the storage size of each reserved storage area, and the total number of reserved storage areas is obtained by (initial number + (initial data amount * numerical value of editing frequency)).

[0085] In some other embodiments, selectively, based on editing frequency, relational data between pre-stored editing frequency and quantity coefficient is retrieved to obtain a target quantity coefficient corresponding to a predetermined editing frequency range in which the editing frequency is located. The required storage area size for the files to be processed is obtained from the product of the target quantity coefficient and the file size of the files to be processed. The storage area size of each predetermined reserved storage area is obtained, and the quantity of reserved storage area for the files to be processed is obtained by the ratio of storage area size / storage area size of each reserved storage area. Alternatively, the storage area size of each predetermined reserved storage area is obtained, and the initial quantity of reserved storage area for the files to be processed is obtained by the ratio of file size of the files to be processed / storage area size of each reserved storage area, calculated as (initial number + (initial data amount * target quantity coefficient)). Here, the relational data between editing frequency and quantity coefficient includes a plurality of predetermined editing frequency ranges and quantity coefficients corresponding to each predetermined editing frequency range.

[0086] In some other embodiments, selectively, the editing frequency and the file size of the files to be processed can be input into a predetermined predictive model to obtain the number of reserved storage areas for the files to be processed. Here, the predictive model may be a mathematical model, such as a linear equation model or an exponential equation model.

[0087] Step 4023: Determine the target reserved storage area for the files to be processed, based on the number of reserved storage areas for the files to be processed.

[0088] Here, the number of target reserved areas in the target reserved area is greater than or equal to the number of reserved storage areas for the files to be processed.

[0089] In some embodiments, selectively, based on the number of reserved storage areas for the files to be processed, the same number of contiguous free storage areas can be determined from the remaining space of the structured file system and set as the target reserved storage area for the files to be processed.

[0090] In some other embodiments, selectively, it is possible to determine whether an existing target reserved area exists for a file to be processed, based on the file identifier of the file to be processed. If no existing target reserved area exists for a file to be processed, the storage size of each predetermined reserved storage area is obtained, the initial number of reserved storage areas for the file to be processed is obtained by the file size of the file to be processed / the storage size of each reserved storage area, the final number of reserved storage areas for the file to be processed is obtained by the number of reserved storage areas for the file to be processed + the initial number, and a contiguous free storage area in the remaining area of ​​the structured file system whose number is the same as the final number is set as the target reserved area for the file to be processed.

[0091] In some other embodiments, if an existing target reserved area exists associated with the file to be processed, the number of free reserved storage spaces in the existing target reserved area is compared with the number of reserved storage spaces in the file to be processed. If the number of free reserved storage spaces in the existing target reserved area is greater than or equal to the number of reserved storage spaces in the file to be processed, the existing target reserved area associated with the file to be processed is set as the target reserved area for the file to be processed. If the number of free reserved storage spaces in the existing target reserved area is less than or equal to the number of reserved storage spaces in the file to be processed, the existing target reserved area is updated to obtain the target reserved area for the file to be processed.

[0092] Here, updating an existing target reserved area may mean increasing the number of free reserved storage areas in the existing target reserved area. In some embodiments, the number of free reserved storage areas in an existing target reserved area can be increased by setting a target number of free storage areas contiguous with the existing target reserved area from the remaining area of ​​the structural file system as free reserved storage areas. In some other embodiments, the number of free reserved storage areas in an existing target reserved area can be increased by merging or erasing data in used reserved storage areas in the existing target reserved area. Here, the target number may be determined based on the difference between the number of free reserved storage areas in the existing target reserved area and the number of reserved storage areas for files to be processed, for example, by setting the absolute value of the difference as the target number, or by setting the sum of the absolute value of the difference and a predetermined parameter as the target number.

[0093] In some other embodiments, if the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, it is possible to determine whether an existing target reserved area exists for the file to be processed based on the file identifier of the file to be processed. If no existing target reserved area exists for the file to be processed, it is determined that a target reserved area exists for the file to be processed based on steps 4021 to 4023 above. However, if an existing target reserved area exists for the file to be processed, the target reserved area for the file to be processed may be determined based on the number of free reserved storage areas in the existing target reserved area, the file size of the file to be processed, the cumulative number of writes, and the creation time.

[0094] In some embodiments, selectively, if an existing target reserved area exists associated with the file to be processed, the number of free reserved storage areas in the existing target reserved area can be compared with a predetermined number of thresholds. If the number of free reserved storage areas in the existing target reserved area is greater than the predetermined number of thresholds, the existing target reserved area associated with the file to be processed is set as the target reserved area for the file to be processed. If the number of free reserved storage areas in the existing target reserved area is less than or equal to the predetermined number of thresholds, the number of reserved storage areas for the file to be processed is determined based on steps 4021 to 4023, and the existing target reserved area is updated according to step 4023 to obtain the target reserved area for the file to be processed based on the number of reserved storage areas for the file to be processed.

[0095] Step 403: Determine the segment order of each file segment based on the placement parameters of each file segment.

[0096] This embodiment can be understood by referring to the contents of step 302 in Figure 3, so the explanation is omitted here.

[0097] Selectively, in some embodiments of the present invention, after determining the target reserved area of ​​the file to be processed, the segment order of each file segment can be obtained by sorting each file segment in ascending order of numerical values ​​based on the configuration parameters of each file segment.

[0098] In some other embodiments of the present invention, an out-of-place update mechanism is considered in which, after one file segment of the structural file segments is updated, the unupdated file segments are not immediately erased, but rather a garbage collection mechanism is periodically triggered to perform garbage collection on file segments that need to be erased in the structural file system. Therefore, if there may be at least two file segments in the structural file system that have the same configuration information but different write times, determining the segment order of each file segment based only on the configuration parameters of each file segment may result in inaccuracies in the segment order. Based on this, in order to improve the accuracy of the segment order of file segments, the segment order of each file segment can be determined based on the configuration parameters and write time of each file segment in the file to be processed. Exemplarily, taking as an example that the configuration information includes the segment code of the file segment, a method for determining the segment order of file segments includes at least steps 4031 to 4033.

[0099] Step 4031: Determine whether each file segment has a first file segment with the same segment code.

[0100] Step 4032: If there are first file segments with the same segment code, determine the time difference between the write time and the current time for each first file segment.

[0101] Step 4033: Sort the first and second file segments with the smallest time difference according to the segment code to obtain the segment order.

[0102] Here, the second file segment is the file segment other than the first file segment within each file segment.

[0103] In some embodiments, if there is no first file segment with the same segment code, the segment order of each file segment is obtained by sorting each file segment in ascending order of code based on the segment code of each file segment.

[0104] In some other embodiments of the present invention, in order to improve the accuracy of the segment order of file segments, it may be determined whether or not an identifier to be deleted exists in each file segment in the file to be processed. If no identifier to be deleted exists in each file segment in the file to be processed, the segment order of each file segment is obtained by sorting each file segment in ascending order of numerical values ​​based on the configuration parameters of each file segment in the file to be processed. However, if an identifier to be deleted exists in a file segment in the file to be processed, the file segment containing the identifier to be deleted is removed, and the segment order is obtained by sorting each remaining file segment in ascending order of numerical values ​​based on the configuration parameters of each remaining file segment in the file to be processed.

[0105] Step 404: Based on the segment order of each file segment, determine the target position of each file segment within the target reserved area.

[0106] In this embodiment, after determining the segment order, the target position in the target reserved area of ​​each file segment can be determined according to the method of step 303 based on the segment order of each file segment, and here, the explanation of redundant content is omitted.

[0107] In some other embodiments of the present invention, after determining the segment order, the target free reserved storage area corresponding to the segment order of each file segment in the target reserved area can be determined based on the storage address of the free reserved storage area in the target reserved area and the segment order of each file segment, and the storage address of the target free reserved storage area corresponding to the segment order of each file segment can be set as the target position of the file segment in the target reserved area. Specifically, as shown in Figure 7, the present invention provides a method for determining the target position of a file segment in the target reserved area, comprising steps 4041 to 4043.

[0108] Step 4041: Obtain the storage address of each target reserved storage area in the target reserved area.

[0109] Here, the target reserved memory area refers to the available reserved memory area within the target reserved memory area.

[0110] Step 4042: Based on the storage address of each reserved storage area, determine the storage order of each target reserved storage area.

[0111] In some embodiments, each target reserved storage area can be sorted in ascending order of address based on the storage address of the target reserved storage area, and the storage order of each target reserved storage area can be determined based on the sorted target reserved storage area information.

[0112] Step 4043: For each file segment, the storage address of the target reserved storage area whose storage order is the same as the segment order of the file segment is determined as the target position in the target reserved storage area of ​​the file segment.

[0113] In some embodiments of the present invention, after determining the target position in the target reserved area for each file segment, each file segment is written to its corresponding target position in the target reserved area based on the target position in the target reserved area for each file segment.

[0114] Step 405: Write each file segment to its corresponding target location in the target reserved area.

[0115] This embodiment can be understood by referring to step 304 in Figure 3, and redundant information will not be explained here.

[0116] In this embodiment, if the file to be processed is a file written to a structured file system, each file segment is written to the target position in the target reserved area of ​​the file segment based on the target position in the target reserved area of ​​the file segment.

[0117] Selectively, in some other embodiments, if the file to be processed is a file written to a structured file system, the storage address of the file segment of the file to be processed is obtained, and each file segment is synchronized to its target position in the target reserved area of ​​the file segment based on the storage address of each file segment and the target position in the target reserved area of ​​each file segment.

[0118] In some embodiments, selectively, a mapping can be established between the logical address of each file segment and its target location in the target reserved area, based on the target location of each file segment in the target reserved area; the storage address of each file segment can be updated; and each file segment can be written to its corresponding target location in the target reserved area.

[0119] In some other embodiments, selectively, the storage address of each file segment is obtained based on the target position in the target reserved area of ​​each file segment; the offset position of each file segment is determined based on the storage address of each file segment and the target position in the target reserved area of ​​the file segment; each file segment is copied based on the offset position of each file segment to obtain a copy file segment of each file segment; and the copy file segment of each file segment is written to the target position in the target reserved area of ​​the file segment based on the offset position of the file segment.

[0120] In some other embodiments of the present invention, if a rewrite operation is performed on all file segments in a file to be processed, the workload increases when the amount of data is large. If the file to be processed is a file written to a structured file system, and the storage addresses of some file segments in the file to be processed are the same as the target positions in the target reserved area of ​​those file segments, then performing a rewrite operation on these file segments as well would invalidate the write operation and increase the workload. Therefore, considering the need to reduce the amount of data that file segments write to the target reserved area, embodiments of the present invention determine the offset position of each file segment based on the storage address of each file segment and the target position in the target reserved area of ​​the file segment, select the target file segment that needs to be moved based on the offset position, and write the target file segment to the target position in the target reserved area of ​​the target file segment. Specifically, the method for writing file segments to the target reserved area includes steps 4051 to 4053.

[0121] Step 4051: Obtain the storage address of each file segment.

[0122] Step 4052: Determine the offset position of each file segment based on the storage address of each file segment and the target position in the target reserved area.

[0123] Here, the offset position represents the positional difference between the storage address of each file segment and the target position in the target reserved area of ​​that file segment.

[0124] Step 4053: Write the target file segment whose offset position is greater than a predetermined offset position to the target position corresponding to the target file segment in the target reserved area.

[0125] In some embodiments, the offset position of each file segment is compared with a predetermined offset position. If the offset position of the file segment is less than or equal to the predetermined offset position, it is indicated that the storage address of the file segment is the same as the target position in the target reserved area of ​​the file segment, and that there is no need to process the file segment. It is then determined whether the offset position of the next file segment is greater than the predetermined offset position. If the offset position of the file segment is greater than the predetermined offset position, it is indicated that the storage address of the file segment is not the same as the target position in the target reserved area of ​​the file segment, and that there is a need to process the file segment. The file segment is then set as a target file segment, each target file segment is synchronized to its target position in the target reserved area, and the target file segment is written to the target position corresponding to the target file segment in the target reserved area.

[0126] As an example, as shown in Figure 8, Figure 8 is a schematic diagram of the internal file fragmentation process in an embodiment of the present invention. For file segments A1, A2, A3, and A4 of file A, as shown in Figure 8(a), file segments A2, A3, and A4 are stored in the existing target reserved area of ​​file A, and the storage addresses of A3 and A4 in file segment A2 are out of order. If file segment A1 is stored in the normal storage area of ​​the structured file system, i.e., if the degree of internal fragmentation of file A is large, the target reserved area of ​​file A is determined based on steps 4021 to 4023, the target position of each of file segments A1, A2, A3, and A4 in the target reserved area is determined based on steps 4041 to 4043, and file segments A1, A2, A3, and A4 are synchronized to the target position of their respective target reserved areas according to steps 4051 to 4053. As shown in Figure 8(b), file segment A1 in storage area 00 is synchronized to target position Y00 in the target reserved area, file segment A2 at position Y05 in the target reserved area is synchronized to target position Y01 in the target reserved area, file segment A3 at position Y06 in the target reserved area is synchronized to target position Y02 in the target reserved area, and file segment A2 at position Y04 in the target reserved area is synchronized to target position Y03 in the target reserved area. Simultaneously, by erasing the file segments in storage area 00, target reserved area Y05, target reserved area Y06, and target reserved area Y04, file segments A1, A2, A3, and A4 are stored consecutively in the target reserved area in segment order, as shown in Figure 8(c).

[0127] In some embodiments of the present invention, after selectively writing each target file segment in the file to be processed to the target location of the target file segment in the target reserved area, the remaining space in the structured file system can be increased by deleting target file segments that are not in the target location in the file to be processed. Alternatively, by setting an identifier to be deleted for target file segments that are not in the target location in the file to be processed, the target file segments with the identifier to be deleted will be deleted when the garbage collection mechanism is triggered next.

[0128] Selectively, in some embodiments of the present invention, after each file segment has been written to its corresponding target location in the target reserved area, if it is detected that a file segment in the file to be processed needs to be updated, it can be updated by an in-place update mechanism. For example, if file segment A1 in the file to be processed needs to be updated, the received file segment A1 to be updated may be newly written to the target location M1 where file segment A1 is located in the target reserved area, overwriting file segment A1 at target location M1.

[0129] Selectively, in some embodiments of the present invention, after each file segment has been written to its corresponding target location in the target reserved area, if it is detected that a file segment in the file to be processed needs to be updated, it can be updated by an out-of-place update mechanism. For example, if it is necessary to update file segment A1 in the file to be processed, the received file segment A1 to be updated is newly written to the first free reserved storage area in the target reserved area, and the file segment A1 stored at target location M1 in the target reserved area is erased, steps 404 and 405 are executed, and the file segment A1 to be updated is newly written to target location M1.

[0130] Step 406: After step 401, if the internal fragmentation data of the file to be processed is less than a predetermined internal fragmentation threshold, the file to be processed is written to the structured file system, or the file to be processed that has been written to the structured file system is not processed.

[0131] Here, writing the file to be processed from the file system to be written to the structured file system may mean writing the file to be processed to the remaining space within the structured file system, or writing the file to be processed to the free reserved storage space within the target reserved area of ​​the structured file system. To understand this, if there is little internal fragmentation data in the file to be processed, the target reserved area from which the file to be processed is written to the structured file system does not write each file segment of the file to be processed to the target location in the corresponding target reserved area.

[0132] In this embodiment of the present invention, the internal fragmentation data of a file determines whether or not to perform internal fragmentation on the file to be processed. If internal fragmentation is necessary for the file to be processed, a target reserved area is determined based on the file size, creation time, and cumulative number of writes. Based on the offset position of each file segment in the file to be processed, each file segment is written to the target reserved area. This ensures that each file segment of the file to be processed is arranged consecutively and sequentially in the target reserved area, thereby reducing internal file fragmentation and mitigating the problem of internal file fragmentation.

[0133] Based on the embodiment shown in Figure 3, in order to improve system performance, reducing the I / O consumption when reading files that need to be read sequentially can be achieved by first determining the file type based on the file attribute information of the file to be processed, before obtaining the configuration parameters of each file segment in the file to be processed, and if the file type is a predetermined target type, determining the target reserved area of ​​the file to be processed. As shown in Figure 9, Figure 9 is a schematic diagram of another embodiment of the file processing method of the present application. Another embodiment of the file processing method of the present application shown in Figure 9 may include the following steps.

[0134] Step 901: Determine the file attribute information of the file to be processed.

[0135] Here, file attribute information includes, but is not limited to, file size, internal fragmentation data, file write count, file read size, and file read parameters. Here, file read parameters include the number of times the file to be processed has been read using the target read method. Here, the target read method includes sequential read.

[0136] In some embodiments, the internal fragmentation data of the file to be processed may be determined based on steps 4011 to 4014 described above.

[0137] Step 902: Determine the file type of the file to be processed based on the file attribute information.

[0138] Here, file types include, but are not limited to, files that are prone to fragmentation, fragmentation-sensitive files, files that are not prone to fragmentation, files that are not prone to fragmentation, and normal files. Here, a normal file is a file whose file type is neither a file that is prone to fragmentation nor a file that is prone to fragmentation. A file that is prone to fragmentation is a file whose memory addresses are not contiguous, and a fragmentation-sensitive file is a file that needs to be read sequentially.

[0139] In some embodiments, fragmentation-prone file identification can determine whether a file type to be processed is fragmentation-prone based on file size, internal fragmentation data, and the number of file writes. Specifically, the fragmentation-prone file determination method includes the contents of steps 9021A to 9022A.

[0140] In step 9021A, the file size is compared with a first predetermined file size threshold, the internal fragmentation data is compared with a predetermined first fragmentation threshold, and the number of file writes is compared with a predetermined file write count threshold.

[0141] In some embodiments, the file size, internal fragmentation data, and number of file writes of the file to be processed may be compared with their respective corresponding thresholds. For example, the file size may be compared with a first predetermined file size threshold, the internal fragmentation data may be compared with a predetermined first fragmentation threshold, and the number of file writes may be compared with a predetermined number of file writes threshold.

[0142] In some other embodiments, the file size, file write count, and internal fragmentation data of the file to be processed may be sequentially compared with corresponding thresholds. For example, the file size of the file to be processed is compared with a first predetermined file size threshold, and if the file size is smaller than the first predetermined file size threshold, it is determined that the file type of the file to be processed is a file that is prone to unfragmentation. If the file size of the file to be processed is equal to or greater than the first predetermined file size threshold, the file write count is compared with a predetermined file write count threshold, and if the file write count is smaller than the predetermined file write count threshold, it is determined that the file type of the file to be processed is a file that is prone to unfragmentation. If the file write count is equal to or greater than the predetermined file write count threshold, the internal fragmentation data is compared with a predetermined first segment threshold, and if the internal fragmentation data is smaller than the predetermined first segment threshold, it is determined that the file type of the file to be processed is a file that is prone to unfragmentation. If the internal fragmentation data is equal to or greater than the predetermined first segment threshold, it is determined that the file type of the file to be processed is a file that is prone to fragmentation.

[0143] In step 9022A, if the file size is greater than or equal to a predetermined file size threshold, the internal fragmentation data is greater than or equal to a predetermined first fragmentation threshold, and the number of file writes is greater than or equal to a predetermined file write count threshold, it is determined that the file type of the file to be processed is a file that is prone to fragmentation.

[0144] In this embodiment, if the file to be processed satisfies one or more of the following conditions, the file type of the file to be processed is determined to be a file that is easily unfragmented: the file size is less than a predetermined file size threshold, the internal fragmentation data is less than a predetermined first fragmentation threshold, and the number of file writes is less than a predetermined file write count threshold.

[0145] In some other embodiments, it is possible to query whether the file to be processed contains fragmentation-prone file marks, and if the file to be processed contains fragmentation-prone file marks, the file type is determined to be a fragmentation-prone file. If the file to be processed does not contain fragmentation-prone file marks, the file type of the file to be processed is determined by performing the fragmentation-prone file determination method shown in steps 9021A to 9022A above.

[0146] In some embodiments, segment file identification determines whether a file type is a segment-sensitive file based on file size, internal fragmentation data, file read size, and file read parameters. Specifically, the method for determining a fragmentation-sensitive file includes steps 9021B to 9022B.

[0147] Step 9021B: The file size is compared with a second predetermined file size threshold, the file read size is compared with a predetermined file read size threshold, the internal fragmentation data is compared with a predetermined second segment threshold, and the file read parameters are compared with predetermined read parameters.

[0148] In step 9022B, if the file size is greater than or equal to a second predetermined file size threshold, the file read size is greater than or equal to a predetermined file read size threshold, the internal fragmentation data is greater than or equal to a predetermined second segment threshold, and the file read parameters are greater than or equal to predetermined read parameters, it is determined that the file type of the file to be processed is a sensitive file that is prone to fragmentation.

[0149] In this embodiment, if the file to be processed satisfies one or more of the following conditions, the file type of the file to be processed is determined to be a non-fragmentation sensitive file: the file size is less than a second predetermined file size threshold, the file read size is less than a predetermined file read size threshold, the internal fragmentation data is less than a predetermined second fragmentation threshold, and the file read parameters are less than predetermined read parameters.

[0150] In some embodiments, it is possible to query whether the file to be processed contains a shard-sensitive file mark, and if the file to be processed contains a shard-sensitive file mark, it is determined that the file type is a shard-sensitive file. If the file to be processed does not contain a shard-sensitive file mark, the file type of the file to be processed is determined by performing the shard-sensitive file determination method shown in steps 9021B to 9022B above.

[0151] Furthermore, the embodiments of this disclosure do not limit the specific numerical values ​​of the predetermined first file size threshold, predetermined first segment threshold, predetermined file write count threshold, predetermined second file size threshold, predetermined second segment threshold, predetermined read parameters, and predetermined file read size threshold, but can be set according to the specific application scenario.

[0152] In some embodiments, the file type of a file to be processed can be determined by executing steps 9021A to 9022A and steps 9021B to 9022B in parallel for the file to be processed. Alternatively, the file type of a file to be processed can be determined by executing steps 9021A to 9022A and steps 9021B to 9022B according to a predetermined execution order. For example, the fragmentation-prone file identification results from steps 9021A to 9022A may be executed first, followed by the fragmentation-sensitive file identification results from steps 9021B to 9022B, and the file type of a file to be processed may be determined based on the fragmentation-prone file identification results and the fragmentation-sensitive file identification results. Alternatively, steps 9021B to 9022B may be executed first to obtain the fragmentation-sensitive file identification results, followed by steps 9021A to 9022A to determine the file type of a file to be processed based on the fragmentation-sensitive file identification results and the fragmentation-sensitive file identification results. Here, the identification result for files prone to fragmentation indicates whether or not the file to be processed is prone to fragmentation.

[0153] Step 903: If the file type is a predetermined target type, obtain the configuration parameters of each file segment in the file to be processed and determine the target reserved area of ​​the file to be processed.

[0154] Here, the target type includes at least one of the following: a file that is prone to fragmentation and a file that is sensitive to fragmentation.

[0155] In some embodiments, when the file type is a predetermined target type, this can be understood by referring to the contents of steps 4021 to 4023 in Figure 6, which are omitted here.

[0156] Step 904: Determine the segment order of each file segment based on the placement parameters of each file segment.

[0157] This embodiment can be understood by referring to the contents of step 403 in Figure 4, but the explanation is omitted here.

[0158] Step 905: Based on the segment order of each file segment, determine the target position of each file segment within the target reserved area.

[0159] This embodiment can be understood by referring to the contents of step 404 in Figure 4, so the explanation is omitted here.

[0160] Step 906: Write each file segment to its corresponding target location in the target reserved area.

[0161] This embodiment can be understood by referring to step 405 in Figure 4, and redundant information will not be explained here.

[0162] If, after steps 907 and 902, the file type is not the target type, the file to be processed is written to the structured filesystem, or the file to be processed that has been written to the structured filesystem is not processed.

[0163] In some embodiments, if the file to be processed is a file written to a structured file system, and the file type of the file to be processed is not a predetermined target type, then internal fragmentation processing is not performed on the file to be processed.

[0164] In some other embodiments, if the file to be processed is a file in a structured file system to be written to, and the file type of the file to be processed is not a predetermined target type determination result, the out-of-place update mechanism writes the file to be processed to the remaining area of ​​the structured file system, or the out-of-place update mechanism writes the file to be processed to the target reserved area of ​​the file to be processed, or the on-site update mechanism writes the file to be processed to the remaining area of ​​the structured file system, or the on-site update mechanism writes the file to be processed to the target reserved area of ​​the file to be processed.

[0165] In an embodiment of the present invention, the file type of the file to be processed is determined based on file attribute information, and if the file type of the file to be processed is a predetermined target type, the target reserved area of ​​the file to be processed is determined and internal fragmentation processing is performed. This allows internal fragmentation processing to be performed on files to be processed that have internal fragmentation data smaller than a predetermined internal fragmentation threshold but need to be read sequentially, thereby reducing I / O consumption when reading such files and improving the performance of the storage device.

[0166] Based on the embodiment shown in Figure 3, in order to further improve system performance and reduce the workload of the structured file system, first, if the file type of the file to be processed is determined to be a predetermined target type according to the method for determining files prone to fragmentation and the method for determining fragmentation-sensitive files, then it is possible to decide whether or not to perform internal fragmentation processing based on the internal fragmentation data of the file to be processed. In this way, by selecting the file type and internal fragmentation data in two rounds, the number of files to be processed that require internal fragmentation processing is reduced, thereby reducing the workload of the structured file system. As shown in Figure 10, another embodiment of the file processing method according to this application may include the following steps.

[0167] Step 1001: Determine the file attribute information of the file to be processed.

[0168] File attribute information includes, but is not limited to, file size, internal fragmentation data, file write count, file read size, and file read parameters. Here, the file read parameters include the number of times the file to be processed has been read using the target read method, where the target read method includes sequential reads.

[0169] In some embodiments, the internal fragmentation data of the file to be processed may be determined based on steps 4011 to 4014 described above.

[0170] Step 1002: Determine the file type of the file to be processed based on the file attribute information.

[0171] Here, file types include, but are not limited to, files that are prone to fragmentation, fragmentation-sensitive files, files that are not prone to fragmentation, files that are not prone to fragmentation, and normal files. Here, a normal file is a file whose file type is neither a file that is prone to fragmentation nor a file that is prone to fragmentation. A file that is prone to fragmentation is a file whose memory addresses are not contiguous, and a fragmentation-sensitive file is a file that needs to be read sequentially.

[0172] In this embodiment, the determination of the file type of the file to be processed in step 902 in Figure 9 will be explained.

[0173] Step 1003: If the file type is a predetermined target type and the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, the system retrieves the configuration parameters of each file segment in the file to be processed and determines the target reserved area of ​​the file to be processed.

[0174] In some embodiments, if the file type is a predetermined target type, the internal fragmentation data in the file attribute information of the file to be processed is compared with a predetermined internal fragmentation threshold. If the internal fragmentation data of the file to be processed is greater than or equal to the predetermined internal fragmentation threshold, the configuration parameters of each file segment in the file to be processed are obtained, and the target reserved area of ​​the file to be processed is determined.

[0175] In some embodiments, when the file type is a predetermined target type and the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, refer to the contents of steps 4021 to 4023 in Figure 6, which are omitted here for explanation.

[0176] Step 1004: Determine the segment order of each file segment based on the placement parameters of each file segment.

[0177] This embodiment can be understood by referring to step 403 in Figure 4, and redundant information will not be explained here.

[0178] Step 1005: Based on the segment order of each file segment, determine the target position of each file segment within the target reserved area.

[0179] This embodiment can be understood by referring to step 404 in Figure 4, and redundant information will not be explained here.

[0180] In step 1006, each file segment is written to its corresponding target location within the target reserved area.

[0181] This embodiment can be understood by referring to step 405 in Figure 4, and redundant information will not be explained here.

[0182] If, after step 1007, step 1002, the internal fragmentation data of the file to be processed is less than a predetermined internal fragmentation threshold, and / or the file type is not the target type, the file to be processed is written to the structured file system, or the file to be processed written to the structured file system is not processed.

[0183] In some embodiments, internal fragmentation processing is not performed on the file to be processed if the file to be processed is a file written to a structured file system, if the internal fragmentation data of the file to be processed is less than a predetermined third fragmentation threshold, or if the file type of the file to be processed is not a predetermined target type.

[0184] In some other embodiments, if the file to be processed is a file in a structured file system to be written to, and the internal fragmentation data of the file to be processed is less than a predetermined internal fragmentation threshold, or the file type of the file to be processed is not a predetermined target type determination result, the out-of-place update mechanism writes the file to be processed to the remaining area of ​​the structured file system, or the out-of-place update mechanism writes the file to be processed to the target reserved area of ​​the file to be processed, or the on-site update mechanism writes the file to be processed to the remaining area of ​​the structured file system, or the on-site update mechanism writes the file to be processed to the target reserved area of ​​the file to be processed.

[0185] According to embodiments of the present invention, by determining whether or not to perform internal fragmentation processing on files to be processed based on the file type and internal fragmentation data of the files to be processed, internal fragmentation processing does not need to be performed on files that are prone to fragmentation but do not need to be read sequentially, thereby reducing the workload of the structured file system and improving system stability.

[0186] Based on the embodiment shown in Figure 3, in order to improve system performance, enhance processing efficiency in the fragmentation process of the structured file system, and reduce the workload of the structured file system, it is possible to decide whether or not to perform internal fragmentation based on the internal fragmentation data of files to be processed from a pre-constructed list of files to be organized. Therefore, by simply deciding whether or not to perform fragmentation on the internal fragmentation data of files to be processed in the list of files to be organized, the processing efficiency in the fragmentation process of the structured file system can be improved, and the number of files to be processed can be reduced by performing screening based on two conditions: the list of files to be organized and the internal fragmentation data of files to be processed.

[0187] As shown in Figure 11, Figure 11 is a schematic diagram of another embodiment of the file processing method according to the present application, and the other embodiment of the file processing method according to the present application shown in Figure 11 may include the following steps.

[0188] Step 1101: Traverse the pre-stored list of files to be sorted, set the currently traversed file in the pre-stored list of files to be sorted as the file to be processed, and determine the internal fragmentation data of the file to be processed.

[0189] In some embodiments, the files to be stored in the list to be sorted are files that have a fragmentation-sensitive file mark and / or a file that is prone to fragmentation. Here, the fragmentation-sensitive file mark is used to indicate that the file type is a fragmentation-sensitive file.

[0190] Selectively, when a file structure filesystem receives a file write request, it obtains file attribute information of the file to be written corresponding to the file write request, determines the file type based on the file attribute information of the file to be written, marks the file to be written and / or a file that is easily fragmented if the file type is a fragmentation-sensitive file and / or a fragmentation-easy file, and updates the list to be organized. Specifically, as shown in Figure 12, Figure 12 is a schematic diagram of an embodiment of the method for updating the list to be organized according to an embodiment of the present application, and the method for updating the list to be organized shown in Figure 12 includes steps 1201 to 1205.

[0191] Step 1201: Respond to a file write request and obtain the file attribute information of the file to be written corresponding to the file write request.

[0192] File attribute information includes, but is not limited to, file size, internal fragmentation data, file write count, file read size, and file read parameters. Here, the file read parameters include the number of times the file to be written has been read using the target read method, where the target read method includes sequential reads.

[0193] In some embodiments, the internal fragmentation data of the file to be written may be determined based on steps 4011 to 4014 above.

[0194] Step 1202: Determine the file type of the file to be written based on the file attribute information of the file to be written.

[0195] Here, file types include, but are not limited to, files that are prone to fragmentation, fragmentation-sensitive files, files that are not prone to fragmentation, files that are not prone to fragmentation, and normal files. Here, a normal file is a file whose file type is neither a file that is prone to fragmentation nor a file that is prone to fragmentation. A file that is prone to fragmentation is a file whose memory addresses are not contiguous, and a fragmentation-sensitive file is a file that needs to be read sequentially.

[0196] This embodiment will be described with reference to determining the file type of the file to be written in step 902 of Figure 9.

[0197] Step 1203: If the file type of the file to be written is the specified target type, mark the file to be written.

[0198] In some embodiments, if the file type of the file to be written is a file that is prone to fragmentation, the file to be written is marked as a file that is prone to fragmentation, and if the file type of the file to be written is a fragmentation-sensitive file, the file to be written is marked as a fragmentation-sensitive file.

[0199] Step 1204: Write the marked files to be written to the target reserved area of ​​the structured file system, and add the file identifiers of the marked files to be written to a pre-stored sorting list.

[0200] In some embodiments, when writing a marked file to be written to a target reserved area of ​​a structured file system, the process of determining the target reserved area for the file to be written based on the file size, cumulative write count, and creation time conditions of the file to be written, by referring to the content conditions in steps 4021 to 4023 in Figure 6, and then writing the file to be written to the target reserved area, is omitted here.

[0201] In some embodiments, when writing a marked file to be written to the target reserved area of ​​the structured file system, it is determined whether an existing target reserved area associated with the file to be written exists in the structured file system. If no existing target reserved area associated with the file to be written exists, the target reserved area for the file to be written is determined by referring to steps 4021 to 4023 in Figure 6, which will not be repeated here. If an existing target reserved area associated with the file to be written exists, it is determined whether the file to be written is marked as a fragment-prone file, and if the file to be written is not marked as a fragment-prone file, the file to be written is written to the remaining area of ​​the structured file system.

[0202] As an example, as shown in Figure 13, Figure 13 is a schematic diagram of a file writing structure file system according to an embodiment of the present invention. As shown in Figure 13(a), when updating file segment A2 of file A, file A is marked as a file prone to fragmentation because the storage areas of file segments A1, A2, A3, and A4 in the normal storage area of ​​file A are discontinuous and the file segments are out of order. When updating file segment A2 of file A, since there is no associated existing target area for file A, the target reserved area of ​​the file to be written is determined based on the contents of steps 4021 to 4023 in Figure 6. As shown in Figure 13(b), the updated file segment A2 is written to the target reserved area Y00, and file segment A2 in storage area 01 is erased. As shown in Figure 12(c), when file segments A3, A4, and A2 are updated sequentially in file A, the updated file segment A3 is written to the target reserved area Y01, the updated file segment A4 is written to the target reserved area Y02, and the re-updated file A2 is written to the target reserved area Y03. File A3 in storage area 10, file A4 in storage area 08, and file A2 in target reserved area Y00 are erased as shown in Figure 13(d). As shown in Figure 13(e), when A4, A2, and A3 of file A are updated sequentially, the re-updated file segment A4 is written to the target reserved area Y04, the re-updated file segment A2 is written to the target reserved area Y05, and the re-updated file segment A3 is written to the target reserved area Y06. As shown in Figure 13, the file processing method according to the present invention reduces the problem of fragmentation of the remaining area due to file updates in the structured file system without affecting the area outside the target reserved area, by performing a file segment update operation in file A after allocating a target reserved area to file A, that is, it reduces the problem of external file fragmentation due to file updates in the structured file system.

[0203] After steps 1205 and 1202, if the file type of the file to be written is not the specified target type, the file to be written is written to the structured file system.

[0204] In this embodiment, by assigning a continuous target reserved area to files to be written that are marked with a file mark prone to fragmentation and / or a fragmentation-sensitive file mark, the problem of external file fragmentation caused by updating and writing to the files to be written can be effectively mitigated.

[0205] In some embodiments, the files to be written can be written to the structured file system by referring to step 406 in Figure 4.

[0206] Step 1102: If the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, the system retrieves the configuration parameters of each file segment in the file to be processed and determines the target reserved area of ​​the file to be processed.

[0207] In some embodiments, if the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, refer to the contents of steps 4021 to 4023 in Figure 6 for understanding, and the explanation is omitted here.

[0208] Step 1103: Determine the segment order of each file segment based on the placement parameters of each file segment.

[0209] This embodiment can be understood by referring to step 403 in Figure 4, and redundant information will not be explained here.

[0210] Step 1104: Based on the segment order of each file segment, determine the target position of each file segment within the target reserved area.

[0211] This embodiment can be understood by referring to step 404 in Figure 4, and redundant information will not be explained here.

[0212] Step 1105: Write each file segment to its corresponding target location in the target reserved area.

[0213] This embodiment can be understood by referring to step 405 in Figure 4, and redundant information will not be explained here.

[0214] Step 1106: After step 1101, if the internal fragmentation data of the file to be processed is less than a predetermined internal fragmentation threshold, the file identifier of the file to be processed is removed from the pre-stored list of files to be sorted, and the next file in the pre-stored list of files to be sorted is traversed.

[0215] Embodiments of the present invention improve the processing efficiency of the structured file system during the fragmentation process simply by deciding whether or not to perform fragmentation on the internal fragmentation data of the files to be processed in the list to be organized. By performing screening based on two conditions—the list to be organized and the internal fragmentation data of the files to be processed—the number of files to be processed is reduced, thereby achieving the effect of reducing the workload of the structured file system.

[0216] To better implement the file processing method according to the embodiment of the present invention, the embodiment of the present invention provides controlling the file processing device based on an embodiment of the file processing method, specifically as shown in Figure 14, which is a schematic diagram of the structure of the file processing device according to the embodiment of the present invention, and the file processing device shown is An acquisition module 1401 for obtaining the configuration parameters of each file segment in the file to be processed and for determining the target reserved area of ​​the file to be processed, A fragment sorting module 1402 for determining the segment order of each file segment based on the placement parameters of each file segment, A fragmentation-sensitive file address mapping module 1403 for determining the target position of each file segment in the target reserved area based on the segment order of each file segment, The system includes an internal anti-fragmentation recovery module 1404 for writing each of the file segments to a target location in the corresponding target reserved area.

[0217] In some embodiments, the configuration parameter includes the segment code of the file segment. The fragment sort module 1402 determines whether there is a first file segment with the same segment code for each file segment, and if there is a first file segment with the same segment code, it determines the time difference between the write time of each first file segment and the current time. The first and second file segments with the smallest time difference according to the segment code are sorted to obtain the segment order, and the second file segment is the file segment other than the first file segment among the file segments.

[0218] In some embodiments, the fragmentation-sensitive file address mapping module 1403 obtains the storage address of each target reserved storage area in the target reserved area, determines the storage order information of each target reserved storage area based on the storage address of each target reserved storage area, and determines the storage address of the target reserved storage area in the target reserved area of ​​each file segment information whose storage order is the same as the segment order of the file segment, as the target position of the file segment.

[0219] In some embodiments, the internal anti-fragmentation recovery module 1404 is for obtaining the storage address of each file segment and determining the offset position of each file segment based on the storage address of each file segment and the target position in the target reserved area. The offset position represents the positional difference between the storage address of each file segment and the target position of the file segment in the target reserved area, and for target file segments whose offset position is greater than a predetermined offset position, the target file segment is written to the target position in the target reserved area of ​​the target file segment.

[0220] In some embodiments, the acquisition module 1401 determines the target reserved area of ​​the file to be processed based on the file size, cumulative number of writes, and creation time of the file to be processed.

[0221] In some embodiments, the control acquisition module 1401 determines the editing frequency of the file to be processed within the interval time based on the time interval between the creation time of the file to be processed and the current time, and the cumulative number of writes to the file to be processed. Based on the editing frequency and the file size of the file to be processed, it determines the number of reserved storage areas for the file to be processed. Based on the number of reserved storage areas for the file to be processed, it determines that the number of target reserved storage areas in the target reserved storage area of ​​the file to be processed is equal to or greater than the number of reserved storage areas for the file to be processed.

[0222] In some embodiments, the file processing device is The system further includes a first determination module 1405 used to determine the internal fragmentation data of a file to be processed, which, if the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, obtains the configuration parameters of each file segment in the file to be processed and determines the target reserved area of ​​the file to be processed.

[0223] In some embodiments, the first decision module 1405 is used to obtain the storage addresses of file segments of the file to be processed, determines the discrete data of the file to be processed based on the storage addresses of each file segment, determines the whole data of the file to be processed based on state difference information between each file segment and its adjacent file segments, and determines the internal fragmentation data of the file to be processed based on the discrete data and whole data of the file to be processed.

[0224] In some embodiments, the first decision module 1405 compares the state information of each file segment with the state information of its adjacent file segments, determines the number of adjacent file segments that do not match the state information of the file segment as the local hole data level of the file segment, and obtains the hole level of the file to be processed based on the local hole level of each file segment.

[0225] In some embodiments, the first decision module 1405 is for determining the segment order of each file segment based on the configuration parameters of each file segment in the file to be processed, for determining the storage order of each file segment based on the storage address of each file segment in the file to be processed, and for determining the discrete data of the file to be processed based on the difference information between the storage order and the segment order of each file segment.

[0226] In some embodiments, the file processing device is The system further includes a second determination module 1406 used to determine the file attribute information of the file to be processed, which determines the file type of the file to be processed based on the file attribute information, and if the file type is the target type, obtains the configuration parameters of each file segment in the file to be processed and determines the target reserved area of ​​the file to be processed.

[0227] In some embodiments, the file attribute information includes file size, internal fragmentation data, and file write count, and the second decision module 1406, The system includes a file fragmentation-prone file determination unit, which compares the file size with a predetermined first file size threshold, compares the internal fragmentation data with a predetermined first fragmentation threshold, and compares the number of file writes with a predetermined number of file writes threshold. If the file size is greater than or equal to the predetermined first file size threshold, the internal fragmentation data is greater than or equal to the predetermined first fragmentation threshold, and the number of file writes is greater than or equal to the predetermined number of file writes threshold, the system determines that the file type of the file to be processed is a file that is prone to fragmentation.

[0228] In some embodiments, the file attribute information includes the file read size, file read parameters, file size, and internal fragmentation data. The second determination module 1406 is The system includes a fragmentation-sensitive file determination unit, which compares the file size with a predetermined second file size threshold, compares the file read size with a predetermined file read size threshold, compares the internal fragmentation data with a predetermined second fragmentation threshold, and compares the file read parameters with predetermined read parameters. If the file read parameters include the number of times the file to be processed is read using a target read method, the system determines that the file type of the file to be processed is a fragmentation-sensitive file if the file size is greater than or equal to a second predetermined file size threshold, the file read size is greater than or equal to a predetermined file read size threshold, the internal fragmentation data is greater than or equal to a predetermined second fragmentation threshold, and the file read parameters are greater than or equal to predetermined read parameters.

[0229] In some embodiments, the file processing device is The third decision module 1407 further includes a third decision module used to determine file attribute information of the file to be processed. The file attribute information includes file size and internal fragmentation data. Based on the file attribute information, it is determined that the target file type of the file to be processed includes at least one of fragment-prone files and fragmentation-sensitive files. If the file type is the target type and the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal segment threshold, the module retrieves the configuration parameters of each file segment in the file to be processed and determines the target reserved area of ​​the file to be processed.

[0230] In the file processing device according to an embodiment of the present invention, the segment order of each file segment is determined by the arrangement parameters of the file segments, and the target position in the target reserved area of ​​each file segment is determined based on the segment order of each file segment, thereby ensuring that each file segment written to the target reserved area is contiguous and further reducing the problem of internal file fragmentation.

[0231] Embodiments of the present invention further provide a file processing device, and Figure 15 is a schematic diagram showing the structure of a file processing device according to an embodiment of the present invention. Specifically, The file processing device may include components such as a processor 1501 having one or more processing cores, a memory 1502 having one or more computer-readable storage media, a power supply 1503, and an input unit 1504. A person skilled in the art will understand that the structure of the file processing device shown in Figure 15 does not constitute a limitation on the file processing device, and that the file processing device may have more or fewer components, or specific combinations of components, or different arrangements of components than those shown in the figure. The processor 1501 is the control center of the file processing device, and it performs various functions of the file processing device and processes data in order to monitor the entire file processing device by connecting various parts of the entire electronic device using various interfaces and lines, running or executing software programs and / or modules stored in memory 1502, and retrieving data stored in memory 1502. Selectively, the processor 1501 may include one or more processing cores. Selectively, the processor 1501 may integrate an application processor and a modem processor. Here, the application processor mainly handles the operating system, user interface, and application programs, etc., and the modem processor mainly handles wireless communication. It should be understood that the above modem processor may not be integrated into the processor 1501.

[0232] Memory 1502 is used to store software programs and modules, and the processor 1501 performs various functional applications and data processing by executing the computer programs and modules stored in memory 1502. Memory 1502 may mainly include a program storage area and a data storage area. Here, the program storage area may store the operating system, computer programs for at least one function (such as an audio playback function or an image playback function), etc., while the data storage area may store data created by the use of a file processing device, etc. Memory 1502 may also include high-speed random access memory and may further include non-volatile memory such as at least one disk memory, flash memory, or other volatile solid-state memory. Accordingly, memory 1502 may also include a memory controller that provides access to memory 1502 by the processor 1501.

[0233] The file processing device may also include a power supply 1503 for supplying power to various components. Preferably, the power supply 1503 may be logically connected to the processor 1501 via a power management system so that functions such as charging, discharging, and power consumption management are implemented via the power management system. The power supply 1503 may include any components such as one or more DC or AC power supplies, a recharge system, a power fault detection circuit, a power converter or inverter, and a power status indicator.

[0234] The file processing device may further include an input unit 1504. The input unit 1504 is used to receive input numerical or character information and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0235] Although not shown, the file processing device may also include a display unit, etc., which will not be repeated here. In this embodiment, the processor 1501 in the file processing device loads executable files corresponding to the processes of one or more computer programs into memory 1502 according to the following instructions, and various functions are realized by executing the computer programs stored in memory 1502 by the processor 1501.

[0236] Obtain the configuration parameters of each file segment in the file to be processed, and determine the target reserved area of ​​the file to be processed. Based on the placement parameters of each file segment, the segment order of each file segment is determined. Based on the segment order of each file segment, the target position in the target reserved area of ​​each file segment is determined. Each file segment is written to its corresponding target location in the aforementioned target reserved area.

[0237] Embodiment 8: Those skilled in the art will understand that all or some steps of the various methods of the above embodiments can be carried out by completing them with a computer program or by controlling the relevant hardware with a computer program, which may be stored on a computer-readable storage medium and, at runtime, may include the processes of each embodiment of the above methods.

[0238] Therefore, embodiments of the present invention provide a storage medium in which a plurality of instructions are stored, and the instructions are loaded by a processor and can execute steps in a file processing method provided in embodiments of the present invention, for example, the following steps.

[0239] Obtain the configuration parameters of each file segment in the file to be processed, and determine the target reserved area of ​​the file to be processed. Based on the placement parameters of each file segment, the segment order of each file segment is determined. Based on the segment order of each file segment, the target position in the target reserved area of ​​each file segment is determined. Each file segment is written to its corresponding target location in the aforementioned target reserved area.

[0240] The specific implementation of each of the above operations may be described by referring to the previous embodiment and will not be repeated here.

[0241] Here, the computer-readable storage medium may include read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and the like.

[0242] Since a computer program stored on the computer-readable storage medium can execute any of the steps of the file processing method provided in the embodiments of this application, the beneficial effects realized by any of the file processing methods provided in the embodiments of the present invention can be achieved. Further details may be found by referring to the embodiments described above, which will not be repeated here.

[0243] The parameter detection method, apparatus, file processing device, and computer storage medium according to embodiments of the present invention have been described in detail above. While specific examples have been used to illustrate the principles and embodiments of the present invention, the above descriptions of embodiments are provided solely as reference for understanding the methods and core concepts of the present invention. Furthermore, while those skilled in the art may modify the embodiments and scope according to the concept of the present invention, in summary, the contents of this specification should not be understood as limitations on the present invention.

Claims

1. A file processing method, wherein the method is Obtain the configuration parameters of each file segment in the file to be processed, and determine the target reserved area of ​​the file to be processed. The segment order of each file segment is determined based on the placement parameters of each file segment, Based on the segment order of each file segment, the target position of each file segment in the target reserved area is determined, A file processing method characterized by including writing each of the aforementioned file segments to the corresponding target location in the aforementioned target reserved area.

2. The configuration parameters include the segment code of the file segment, and the segment order of each file segment is determined based on the configuration parameters of each file segment. To determine whether a first file segment with the same segment code exists in each of the aforementioned file segments, If there are first file segments with the same segment code, determine the time difference between the write time and the current time for each of the first file segments. The file processing method according to claim 1, characterized in that it includes sorting the first file segment with the smallest time difference and the second file segment, which is a file segment other than the first file segment in each of the file segments, according to the segment code to obtain the segment order.

3. Determining the target position of each file segment in the target reserved area based on the segment order of each file segment is: To obtain the storage address of each target reserved storage area in the aforementioned target reserved area, Based on the storage address of each of the aforementioned target reserved storage areas, the storage order of each of the aforementioned target reserved storage areas is determined. The file processing method according to claim 1, characterized in that for each file segment, the storage address of a target reserved storage area whose storage order is the same as the segment order of the file segment is determined as the target position of the file segment in the target reserved storage area.

4. After determining the target position in the target reserved area of ​​each file segment based on the segment order of each file segment, the method then: To obtain the storage address of each of the aforementioned file segments, The offset position of each file segment is determined based on the storage address of each file segment and the target position in the target reserved area. The file processing method according to claim 1, wherein for a target file segment whose offset position is greater than a predetermined offset position, the method includes writing the target file segment to the target position in the target reserved area of ​​the target file segment, wherein the offset position represents the positional difference between the storage address of each file segment and the target position in the target reserved area of ​​the file segment.

5. Determining the target reserved area of ​​the file to be processed is The file processing method according to claim 1, characterized in that it includes determining a target reserved area for the file to be processed based on the file size, cumulative number of writes, and creation time of the file to be processed.

6. Determining the target reserved area of ​​the file to be processed based on the file size, cumulative number of writes, and creation time of the file to be processed is: Based on the time interval between the creation time of the file to be processed and the current time, and the cumulative number of writes to the file to be processed, the frequency of editing the file to be processed within the time interval is determined. Based on the editing frequency and the file size of the files to be processed, the number of reserved storage areas for the files to be processed is determined. The file processing method according to claim 5, characterized in that it includes determining a target reserved storage area for a file to be processed, such that the number of target reserved storage areas for the file to be processed is equal to or greater than the number of reserved storage areas for the file to be processed, based on the number of reserved storage areas for the file to be processed.

7. Before obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed, the method: Determining the internal fragmentation data of the file to be processed, The file processing method according to any one of claims 1 to 6, characterized in that if the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, the method is further characterized in that it obtains the configuration parameters of each file segment in the file to be processed and determines the target reserved area of ​​the file to be processed.

8. Determining the internal fragmentation data of the file to be processed is: Obtaining the storage address of the file segment of the file to be processed, Based on the storage address of each file segment, the discrete data of the file to be processed is determined, Based on the state difference information between each file segment and its adjacent file segments, the whole data of the file to be processed is determined. The file processing method according to claim 7, characterized in that it includes determining the acquisition of internal fragmentation data of the file to be processed based on the discrete data and whole data of the file to be processed.

9. Determining the whole data of the file to be processed based on state difference information between each file segment and its adjacent file segments is: For each file segment, the state information of the file segment is compared with the state information of the adjacent file segments of the file segment. The number of adjacent file segments that do not match the state information of the file segment is determined as the local hole data of the file segment, The file processing method according to claim 8, further comprising obtaining the hole level of the file to be processed based on the local hole level of each of the file segments.

10. Determining the discrete data of the file to be processed based on the storage address of each file segment is: Based on the configuration parameters of each file segment in the file to be processed, the segment order of each file segment is determined. Based on the storage address of each file segment in the file to be processed, the storage order of each file segment is determined. The file processing method according to claim 8, characterized in that it includes determining the acquisition of discrete data of the file to be processed based on difference information between the storage order of each file segment and the segment order.

11. Before obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed, the method: Determining the file attribute information of the file to be processed, Based on the aforementioned file attribute information, the file type of the file to be processed is determined, The file processing method according to any one of claims 1 to 6, characterized in that, if the file type is a target type that includes at least one of a file that is prone to fragmentation and a fragmentation-sensitive file, the method includes obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed.

12. The aforementioned file attribute information includes file size, internal fragmentation data, and file write count. Determining the file type of the file to be processed based on the aforementioned file attribute information is: The process involves comparing the file size with a predetermined first file size threshold, comparing the internal fragmentation data with a predetermined first fragmentation threshold, and comparing the number of file writes with a predetermined file write count threshold. The file processing method according to claim 11, further comprising determining that the file type of the file to be processed is a file that is prone to fragmentation if the file size is equal to or greater than the predetermined first file size threshold, the internal fragmentation data is equal to or greater than the predetermined first fragmentation threshold, and the number of file writes is equal to or greater than the predetermined number of file writes threshold.

13. The aforementioned file attribute information includes file read size, file read parameters, file size, and internal fragmentation data. Determining the file type of the file to be processed based on the aforementioned file attribute information is: The process involves comparing the file size with a predetermined second file size threshold, comparing the file read size with a predetermined file read size threshold, comparing the internal fragmentation data with a predetermined second fragmentation threshold, and comparing the file read parameters, including the number of times the file to be processed is read using the target read method, with predetermined read parameters. The file processing method according to claim 11, characterized in that if the file size is greater than or equal to the predetermined second file size threshold, the file read size is greater than or equal to the predetermined file read size threshold, the internal fragmentation data is greater than or equal to the predetermined second fragmentation threshold, and the file read parameters are greater than or equal to the predetermined read parameters, then it is determined that the file type of the file to be processed is a fragmentation-sensitive file.

14. Before obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed, the method: Determining the file attribute information of the file to be processed, including the file size and internal fragmentation data, Based on the aforementioned file attribute information, the file type of the file to be processed is determined, The file processing method according to any one of claims 1 to 6, characterized in that, if the file type of the file to be processed is a target file type that includes at least one of a file that is prone to fragmentation and a fragmentation-sensitive file, and the internal fragmentation data of the file to be processed is greater than or equal to a predetermined internal fragmentation threshold, the method includes obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed.

15. Obtaining the configuration parameters of each file segment in the file to be processed is: When a file write request is received, trigger the file write operation, The file processing method according to claim 1, characterized by comprising obtaining a file to be processed, and traversing the file segments of the file to be processed to obtain the configuration parameters of each file segment.

16. Obtaining the file to be processed and traversing the file segments of the file to be processed to obtain the configuration parameters of each file segment is: If the aforementioned file write operation is the first write to the file, the file to be written is set as the file to be processed, and the configuration parameters of each file segment of the file to be processed are determined by traversing each file segment. If the aforementioned file write operation is an update of a file segment, the system determines the written file segment in the structured file system that matches the file identifier of the file segment to be updated, sets the set consisting of the file segment to be updated and the file segments in the written file that match the file identifier as the file to be processed, and traverses each file segment in the file to be processed to determine the configuration parameters of each file segment. The file processing method according to claim 15, characterized in that, if the file writing operation is a file update, it includes obtaining the file to be updated, setting the file to be updated as the file to be processed, and traversing each file segment in the file to be processed to determine the configuration parameters of each file segment.

17. Determining the target reserved area of ​​the file to be processed is Obtaining the file identifier of the file to be processed, The file processing method according to claim 1, characterized in that it includes determining a reserved area that matches the file identifier in the structured file system as the target reserved area of ​​the file to be processed.

18. A file processing device, wherein the device is An acquisition module for obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed, A fragment sorting module for determining the segment order of each file segment based on the placement parameters of each file segment, A fragmentation-sensitive file address mapping module for determining the target position in the target reserved area of ​​each file segment based on the segment order of each file segment, A file processing apparatus characterized by including an internal anti-fragmentation recovery module for writing each of the file segments to a target location in the corresponding target reserved area.

19. A file processing device comprising a memory and a processor, wherein an application program is stored in the memory, and the processor is used to execute the application program in the memory in order to perform the operation according to any one of claims 1 to 17.

20. A storage medium, wherein a plurality of commands are stored in the storage medium, and the commands are adapted to be loaded by a processor to perform the steps in the method according to any one of claims 1 to 17.

Images